2019-11-29T10:46:10.243 INFO:root:teuthology version: 1.0.0-e749e7b 2019-11-29T10:46:10.244 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.AqUhhp.tmp -- --verbose --lock --description rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi --block --owner scheduled_yuvalif@teuthology --archive /home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875 2019-11-29T10:46:10.306 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-29T10:46:10.370 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875 branch: wip-yuval-master description: rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: yuvalif@yahoo.com first_in_suite: false job_id: '4552875' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-yuval-master ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd fast shutdown: false osd objectstore: filestore osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: notcmalloc sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 rgw: compression type: random ec-data-pool: true frontend: civetweb storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true workunit: branch: wip-yuval-master sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 owner: scheduled_yuvalif@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 suite: rgw:verify suite_branch: wip-yuval-master suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.9251 2019-11-29T10:46:10.370 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa; will attempt to use it 2019-11-29T10:46:10.371 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa/tasks 2019-11-29T10:46:10.371 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-11-29T10:46:10.373 INFO:teuthology.task.internal:Checking packages... 2019-11-29T10:46:10.386 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '656c8e8049c2c1acd363143c842c2edf1fe09b64' 2019-11-29T10:46:10.386 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-11-29T10:46:10.386 INFO:teuthology.packaging:ref: None 2019-11-29T10:46:10.387 INFO:teuthology.packaging:tag: None 2019-11-29T10:46:10.387 INFO:teuthology.packaging:branch: wip-yuval-master 2019-11-29T10:46:10.387 INFO:teuthology.packaging:sha1: 656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T10:46:10.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-yuval-master 2019-11-29T10:46:10.579 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7809.g656c8e8 2019-11-29T10:46:10.579 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-11-29T10:46:10.597 INFO:teuthology.task.internal:no buildpackages task found 2019-11-29T10:46:10.597 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-11-29T10:46:10.614 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-11-29T10:46:10.615 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-29T10:46:10.678 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:46:20.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:46:30.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:46:40.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:46:50.767 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:00.784 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:10.802 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:20.818 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:30.834 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:40.856 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:47:50.885 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:48:00.900 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:48:10.916 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:48:20.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:48:30.948 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:48:40.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 0)... 2019-11-29T10:48:50.982 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 0)... 2019-11-29T10:49:00.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:49:11.014 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 0)... 2019-11-29T10:49:21.040 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:49:31.058 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:49:41.077 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:49:51.098 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:01.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:11.127 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:21.179 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:31.199 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:41.215 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:50:51.258 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:01.272 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:11.299 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:21.317 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:31.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:41.356 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:51:51.370 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:01.388 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:11.421 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:21.437 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:31.456 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:41.488 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:52:51.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:01.529 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:11.546 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:21.565 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:31.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:41.599 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:53:51.617 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:01.632 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:11.648 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:21.664 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:31.687 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:41.714 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:54:51.730 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:55:01.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-29T10:55:11.778 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_yuvalif@teuthology', 'description': '/home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875', 'machine_type': 'smithi'} 2019-11-29T10:55:11.795 DEBUG:teuthology.lock.ops:locked ubuntu@smithi163.front.sepia.ceph.com, ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T10:55:12.010 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-29T10:55:12.220 INFO:teuthology.provision.fog.smithi069:Scheduling deploy of centos 7.6 2019-11-29T10:55:12.224 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 7.6 2019-11-29T10:55:12.555 INFO:teuthology.orchestra.console:Power off smithi069 2019-11-29T10:55:12.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-29T10:55:12.614 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:12.709 INFO:teuthology.orchestra.console:Power off smithi163 2019-11-29T10:55:12.710 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-29T10:55:12.755 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:16.744 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:16.799 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:20.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:20.934 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:25.010 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:25.088 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:25.249 INFO:teuthology.orchestra.console:Power off for smithi069 completed 2019-11-29T10:55:25.322 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2019-11-29T10:55:25.352 INFO:teuthology.orchestra.console:Power on smithi069 2019-11-29T10:55:25.352 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-29T10:55:25.413 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:25.441 INFO:teuthology.orchestra.console:Power on smithi163 2019-11-29T10:55:25.441 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-29T10:55:25.495 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:29.475 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:29.555 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:29.709 INFO:teuthology.orchestra.console:Power on for smithi069 completed 2019-11-29T10:55:29.810 INFO:teuthology.provision.fog.smithi069:Waiting for deploy to finish 2019-11-29T10:55:33.685 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-29T10:55:33.921 INFO:teuthology.orchestra.console:Power on for smithi163 completed 2019-11-29T10:55:34.022 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2019-11-29T10:58:15.876 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:24.947 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:34.034 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:35.187 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:43.121 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:44.266 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:52.209 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:58:53.326 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:01.293 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:02.381 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:10.349 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:11.434 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:19.405 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:20.529 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:28.529 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:29.613 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:37.617 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:38.670 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:47.725 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T10:59:56.781 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:00:43.660 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:00:43.900 INFO:teuthology.orchestra.run.smithi069:> hostname 2019-11-29T11:00:44.358 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 2019-11-29T11:00:44.359 INFO:teuthology.orchestra.run.smithi069:> grep smithi069 /etc/hosts 2019-11-29T11:00:44.451 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69 smithi069.front.sepia.ceph.com smithi069 2019-11-29T11:00:44.451 INFO:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2019-11-29T11:00:44.607 INFO:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069/smithi069/g' /etc/hosts 2019-11-29T11:00:44.720 INFO:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/172.21.15.69/172.21.15.69/g' /etc/hosts 2019-11-29T11:00:44.808 INFO:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069/smithi069/g' /etc/hostname 2019-11-29T11:00:44.893 INFO:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2019-11-29T11:00:44.966 INFO:teuthology.provision.fog.smithi069:Deploy complete! 2019-11-29T11:00:44.968 DEBUG:teuthology.parallel:result is None 2019-11-29T11:01:02.798 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:01:02.975 INFO:teuthology.orchestra.run.smithi163:> hostname 2019-11-29T11:01:03.504 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com 2019-11-29T11:01:03.504 INFO:teuthology.orchestra.run.smithi163:> grep smithi163.front.sepia.ceph.com /etc/hosts 2019-11-29T11:01:03.586 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2019-11-29T11:01:03.587 INFO:teuthology.orchestra.run.smithi163:> sudo hostname smithi163 2019-11-29T11:01:03.755 INFO:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hosts 2019-11-29T11:01:03.874 INFO:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/172.21.15.163/172.21.15.163/g' /etc/hosts 2019-11-29T11:01:03.965 INFO:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hostname 2019-11-29T11:01:04.048 INFO:teuthology.orchestra.run.smithi163:> sudo hostname smithi163 2019-11-29T11:01:04.125 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2019-11-29T11:01:04.127 DEBUG:teuthology.parallel:result is None 2019-11-29T11:01:04.366 INFO:teuthology.lock.ops:Checking smithi069.front.sepia.ceph.com 2019-11-29T11:01:04.367 INFO:teuthology.lock.ops:New key found. Updating... 2019-11-29T11:01:04.386 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2019-11-29T11:01:04.386 INFO:teuthology.lock.ops:New key found. Updating... 2019-11-29T11:01:04.405 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: set os type and version on server 2019-11-29T11:01:04.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:01:04.406 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:01:04.506 INFO:teuthology.orchestra.run.smithi069:> true 2019-11-29T11:01:04.918 INFO:teuthology.orchestra.run.smithi069:> uname -m 2019-11-29T11:01:04.982 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2019-11-29T11:01:04.983 INFO:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2019-11-29T11:01:05.041 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Linux" 2019-11-29T11:01:05.041 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="7 (Core)" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="7" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-29T11:01:05.042 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.centos.org/" 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-29T11:01:05.043 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-29T11:01:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:05.045 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2019-11-29T11:01:05.061 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2019-11-29T11:01:05.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:01:05.062 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:01:05.165 INFO:teuthology.orchestra.run.smithi163:> true 2019-11-29T11:01:05.559 INFO:teuthology.orchestra.run.smithi163:> uname -m 2019-11-29T11:01:05.643 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2019-11-29T11:01:05.643 INFO:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2019-11-29T11:01:05.713 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Linux" 2019-11-29T11:01:05.713 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="7 (Core)" 2019-11-29T11:01:05.713 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2019-11-29T11:01:05.713 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="7" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.centos.org/" 2019-11-29T11:01:05.714 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-29T11:01:05.715 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:01:05.715 INFO:teuthology.orchestra.run.smithi163.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-29T11:01:05.716 INFO:teuthology.orchestra.run.smithi163.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-29T11:01:05.716 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-29T11:01:05.716 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-29T11:01:05.716 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:01:05.717 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2019-11-29T11:01:05.737 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-11-29T11:01:05.769 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi069.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDiH4AntPmRDAGRgVfFgbbYgSMcmsR5Nk76bDfDRzJf4zB+Ri4450gZKaE2j2bZeSCv0Wc/rj2kYGwZkoOEzZkF6cRoqogAzweakrGC35FqyWhBpnk1zsgGSdyLmTQBlDNd0EdgmFGBWev0y9DkQbMEAzOfQCkvMcAudZiqU2sRZ0lzIPXlTA0V37WFtW7QYI0F6JTKHvZmHtkRpsQ5otM+gRK+Jic8CMdnxokAs/QbxqWr4sK6aM/wEMMr3O/pv3t36PyJ0pjyQ9OmBh+TKHMJgYcX/zqGk6L4vzd861vV2IDg0eIEfuEyoXhbnxIbvhJKWChZ+XBXjekDxcDiwIrJ smithi163.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1Dq5pttXqbPFyjKdzabTdCoOCSLFlDCctS6la1WFsZMHJXrdPYg5Qn6zAPG1mH7WZA+MABKvLCAcDi1PcfX2ex5XDUycR2T1mjYNkvW8uhL3nhiT+NKE9wJ6XMKV63te3qcjcNVVAnXgVW1ptc35VghTmHn6lQqJYSe6sTD6+TkwgmGdtOpm7gUFDPO9m/zCt2exAZVtBzH1UzRfR4qVYlulV6eay5sZXxE/y63/Y9bEKy45IRZp8/pHmqn9D+NME7B0ZG46TEEX06BaGszsQAoqzwqX6ggvhJyneSUxqMCqc24BOi36t18ILaQMcbNVZz9Ell1gMlmO4O/1/TNS1 2019-11-29T11:01:05.770 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-29T11:01:05.822 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-11-29T11:01:05.835 INFO:teuthology.task.internal:Saving configuration 2019-11-29T11:01:05.887 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-11-29T11:01:05.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-11-29T11:01:05.917 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-29 10:55:11.785764', u'locked_by': u'scheduled_yuvalif@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi069.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKz10I5ZV4fojjSKSq7QUNAFvE9lBljcmJZuH7tJcILk9WBIykENFcn9e5a5NKx3iJOGidObjzVAGTkO6DeRMjg=', u'description': u'/home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875'} 2019-11-29T11:01:05.931 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-29 10:55:11.785745', u'locked_by': u'scheduled_yuvalif@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi163.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBVUfcbkmkk8f0JLfX+urxNMH9vnGdpVnvnxzRezodPPc84VuGWF5Fy+bHO3ouY2grzW9aC8W/yvGGyNSdilJHc=', u'description': u'/home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875'} 2019-11-29T11:01:05.931 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-11-29T11:01:05.944 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-11-29T11:01:05.944 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-11-29T11:01:05.945 INFO:teuthology.run_tasks:Running task console_log... 2019-11-29T11:01:06.149 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-29T11:01:06.149 INFO:teuthology.run_tasks:Running task internal.connect... 2019-11-29T11:01:06.159 INFO:teuthology.task.internal:Opening connections... 2019-11-29T11:01:06.160 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:01:06.161 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:01:06.305 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:01:06.306 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:01:06.483 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-11-29T11:01:06.504 INFO:teuthology.orchestra.run.smithi069:> uname -m 2019-11-29T11:01:06.591 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2019-11-29T11:01:06.591 INFO:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2019-11-29T11:01:06.648 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Linux" 2019-11-29T11:01:06.648 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="7 (Core)" 2019-11-29T11:01:06.649 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2019-11-29T11:01:06.649 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2019-11-29T11:01:06.649 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="7" 2019-11-29T11:01:06.649 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-29T11:01:06.650 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2019-11-29T11:01:06.650 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-29T11:01:06.650 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.centos.org/" 2019-11-29T11:01:06.650 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-29T11:01:06.650 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:06.651 INFO:teuthology.orchestra.run.smithi069.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-29T11:01:06.651 INFO:teuthology.orchestra.run.smithi069.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-29T11:01:06.651 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-29T11:01:06.651 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-29T11:01:06.652 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:06.652 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2019-11-29T11:01:06.689 INFO:teuthology.orchestra.run.smithi163:> uname -m 2019-11-29T11:01:06.810 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2019-11-29T11:01:06.810 INFO:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2019-11-29T11:01:06.876 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Linux" 2019-11-29T11:01:06.876 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="7 (Core)" 2019-11-29T11:01:06.876 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2019-11-29T11:01:06.877 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2019-11-29T11:01:06.877 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="7" 2019-11-29T11:01:06.877 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-29T11:01:06.877 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2019-11-29T11:01:06.878 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-29T11:01:06.878 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.centos.org/" 2019-11-29T11:01:06.878 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-29T11:01:06.881 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:01:06.881 INFO:teuthology.orchestra.run.smithi163.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-29T11:01:06.882 INFO:teuthology.orchestra.run.smithi163.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-29T11:01:06.882 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-29T11:01:06.882 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-29T11:01:06.882 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:01:06.883 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2019-11-29T11:01:06.897 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-11-29T11:01:06.941 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-11-29T11:01:06.970 INFO:teuthology.task.internal:Checking for old test directory... 2019-11-29T11:01:06.970 INFO:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2019-11-29T11:01:06.979 INFO:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2019-11-29T11:01:07.058 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-11-29T11:01:07.078 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-11-29T11:01:07.078 INFO:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2019-11-29T11:01:07.084 INFO:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2019-11-29T11:01:07.181 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-11-29T11:01:07.242 INFO:teuthology.run_tasks:Running task kernel... 2019-11-29T11:01:07.294 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-11-29T11:01:07.295 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-11-29T11:01:07.295 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:01:07.296 INFO:teuthology.orchestra.run.smithi069:> sudo package-cleanup -y --oldkernels 2019-11-29T11:01:08.176 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:01:10.177 INFO:teuthology.orchestra.run.smithi069.stdout:No old kernels to remove 2019-11-29T11:01:10.184 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:01:10.185 INFO:teuthology.orchestra.run.smithi163:> sudo package-cleanup -y --oldkernels 2019-11-29T11:01:11.117 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:01:12.980 INFO:teuthology.orchestra.run.smithi163.stdout:No old kernels to remove 2019-11-29T11:01:12.995 INFO:teuthology.orchestra.run.smithi069:> uname -r 2019-11-29T11:01:13.106 INFO:teuthology.orchestra.run.smithi069.stdout:3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:01:13.107 INFO:teuthology.task.kernel:Running kernel on smithi069: 3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:01:13.107 INFO:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2019-11-29T11:01:13.416 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:01:13.486 INFO:teuthology.orchestra.run.smithi069.stdout:Determining fastest mirrors 2019-11-29T11:01:13.656 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.math.princeton.edu 2019-11-29T11:01:13.656 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: ftp.linux.ncsu.edu 2019-11-29T11:01:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:01:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.datto.com 2019-11-29T11:01:15.770 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:01:15.771 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:01:15.771 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-29T11:01:17.085 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:01:17.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:17.958 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:01:17.969 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 50 M 2019-11-29T11:01:17.970 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 64 M 2019-11-29T11:01:17.971 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:01:17.972 INFO:teuthology.orchestra.run.smithi069.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-29T11:01:18.976 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:01:19.170 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:01:20.124 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:01:20.124 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:01:55.628 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-29T11:01:56.126 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-29T11:01:56.126 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:56.127 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:01:56.127 INFO:teuthology.orchestra.run.smithi069.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-29T11:01:56.127 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:01:56.127 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:01:56.173 INFO:teuthology.orchestra.run.smithi069:> rpm -q kernel --last | head -n 1 2019-11-29T11:01:56.579 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Fri 29 Nov 2019 11:01:23 AM UTC 2019-11-29T11:01:56.579 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:01:56.580 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:01:56.580 INFO:teuthology.orchestra.run.smithi163:> uname -r 2019-11-29T11:01:56.699 INFO:teuthology.orchestra.run.smithi163.stdout:3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:01:56.699 INFO:teuthology.task.kernel:Running kernel on smithi163: 3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:01:56.699 INFO:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2019-11-29T11:01:57.015 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:01:57.060 INFO:teuthology.orchestra.run.smithi163.stdout:Determining fastest mirrors 2019-11-29T11:01:57.220 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.math.princeton.edu 2019-11-29T11:01:57.221 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: ftp.linux.ncsu.edu 2019-11-29T11:01:57.222 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:01:57.222 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.datto.com 2019-11-29T11:01:59.432 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:01:59.433 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:01:59.433 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-29T11:02:00.741 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:02:01.626 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:01.626 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:02:01.633 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:01.633 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:02:01.633 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:02:01.633 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2019-11-29T11:02:01.634 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:01.635 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 50 M 2019-11-29T11:02:01.635 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 64 M 2019-11-29T11:02:01.635 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:02:01.640 INFO:teuthology.orchestra.run.smithi163.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-29T11:02:02.601 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:02:02.795 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:02:03.668 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:02:03.668 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:02:39.610 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-29T11:02:40.120 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-29T11:02:40.120 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:40.120 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:02:40.121 INFO:teuthology.orchestra.run.smithi163.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-29T11:02:40.121 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:40.121 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:02:40.176 INFO:teuthology.orchestra.run.smithi163:> rpm -q kernel --last | head -n 1 2019-11-29T11:02:40.601 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Fri 29 Nov 2019 11:02:06 AM UTC 2019-11-29T11:02:40.601 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:40.601 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:40.601 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-11-29T11:02:40.601 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-29T11:02:40.602 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-29T11:02:40.602 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-11-29T11:02:40.602 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2019-11-29T11:02:40.602 INFO:teuthology.orchestra.run.smithi069:> rpm -q kernel --last | head -n 1 2019-11-29T11:02:40.974 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Fri 29 Nov 2019 11:01:23 AM UTC 2019-11-29T11:02:40.975 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:40.976 INFO:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2 2019-11-29T11:02:41.125 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2 2019-11-29T11:02:41.125 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2019-11-29T11:02:41.126 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.02 2019-11-29T11:02:41.126 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 0.76.el7.centos 2019-11-29T11:02:41.126 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2019-11-29T11:02:41.126 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-11-29T11:02:41.127 INFO:teuthology.orchestra.run.smithi069.stdout:Group : System Environment/Base 2019-11-29T11:02:41.127 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 0 2019-11-29T11:02:41.127 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2019-11-29T11:02:41.127 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-29T11:02:41.127 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Relocations : (not relocatable) 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : CentOS BuildSystem 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-29T11:02:41.128 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2019-11-29T11:02:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:02:41.129 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-29T11:02:41.129 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-29T11:02:41.129 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-29T11:02:41.129 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2019-11-29T11:02:41.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-29T11:02:41.130 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:41.130 INFO:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-29T11:02:41.610 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2019-11-29T11:02:41.743 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-29T11:02:41.976 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:02:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-11-29T11:02:42.213 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-29T11:02:42.213 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-29T11:02:42.213 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-11-29T11:02:42.214 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-11-29T11:02:43.292 INFO:teuthology.orchestra.run.smithi069.stderr:done 2019-11-29T11:02:43.295 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:02:43.353 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.ep7BEbnqAr 2019-11-29T11:02:43.353 INFO:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ep7BEbnqAr 2019-11-29T11:02:43.423 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.ep7BEbnqAr 2019-11-29T11:02:43.548 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.ep7BEbnqAr is 6KB 2019-11-29T11:02:43.625 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.ep7BEbnqAr 2019-11-29T11:02:43.721 INFO:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 0 2019-11-29T11:02:43.920 INFO:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2019-11-29T11:02:43.925 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-11-29T11:02:43.925 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2019-11-29T11:02:43.925 INFO:teuthology.orchestra.run.smithi163:> rpm -q kernel --last | head -n 1 2019-11-29T11:02:44.288 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Fri 29 Nov 2019 11:02:06 AM UTC 2019-11-29T11:02:44.289 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:44.289 INFO:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2 2019-11-29T11:02:44.407 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.02 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 0.76.el7.centos 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-11-29T11:02:44.408 INFO:teuthology.orchestra.run.smithi163.stdout:Group : System Environment/Base 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 0 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-29T11:02:44.409 INFO:teuthology.orchestra.run.smithi163.stdout:Relocations : (not relocatable) 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : CentOS BuildSystem 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : CentOS 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:02:44.410 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-29T11:02:44.411 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-29T11:02:44.411 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-29T11:02:44.411 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2019-11-29T11:02:44.413 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-29T11:02:44.413 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:44.413 INFO:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-29T11:02:44.862 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2019-11-29T11:02:45.008 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:02:45.024 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-29T11:02:45.257 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-11-29T11:02:45.273 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-11-29T11:02:45.305 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-29T11:02:45.323 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-29T11:02:45.353 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-11-29T11:02:45.353 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-11-29T11:02:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:done 2019-11-29T11:02:46.473 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:02:46.537 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.7XptHZ0ROm 2019-11-29T11:02:46.537 INFO:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7XptHZ0ROm 2019-11-29T11:02:46.614 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.7XptHZ0ROm 2019-11-29T11:02:46.756 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.7XptHZ0ROm is 6KB 2019-11-29T11:02:46.816 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.7XptHZ0ROm 2019-11-29T11:02:46.864 INFO:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 0 2019-11-29T11:02:47.159 INFO:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2019-11-29T11:02:47.169 INFO:teuthology.misc:Re-opening connections... 2019-11-29T11:02:47.169 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:02:47.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:02:47.170 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:21.784 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:03:21.785 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:03:21.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:21.785 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:40.216 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2019-11-29T11:03:40.217 DEBUG:teuthology.misc:waited 53.0472700596 2019-11-29T11:03:41.217 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:03:41.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:41.218 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:44.281 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:03:44.282 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:03:44.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:44.282 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:47.352 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2019-11-29T11:03:47.353 DEBUG:teuthology.misc:waited 60.1834020615 2019-11-29T11:03:48.353 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:03:48.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:48.354 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:51.416 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:03:51.417 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:03:51.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:51.417 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:54.488 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2019-11-29T11:03:54.489 DEBUG:teuthology.misc:waited 67.3192591667 2019-11-29T11:03:55.489 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:03:55.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:55.490 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:03:58.556 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:03:58.557 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:03:58.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:03:58.557 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:01.624 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2019-11-29T11:04:01.625 DEBUG:teuthology.misc:waited 74.455283165 2019-11-29T11:04:02.625 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:04:02.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:04:02.626 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:05.688 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:04:05.689 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:04:05.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:04:05.689 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:08.760 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2019-11-29T11:04:08.761 DEBUG:teuthology.misc:waited 81.5912640095 2019-11-29T11:04:09.761 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:04:09.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:04:09.762 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:41.156 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2019-11-29T11:04:41.157 INFO:teuthology.misc:trying to connect to ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:04:41.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:04:41.157 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:56.608 INFO:teuthology.orchestra.run.smithi163:> true 2019-11-29T11:04:56.905 DEBUG:teuthology.misc:waited 129.735686064 2019-11-29T11:04:57.908 INFO:teuthology.misc:trying to connect to ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:04:57.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:04:57.909 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:04:58.036 INFO:teuthology.orchestra.run.smithi069:> true 2019-11-29T11:04:58.470 DEBUG:teuthology.misc:waited 131.300986052 2019-11-29T11:04:59.471 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-11-29T11:04:59.472 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.4.3.el7.x86_64"... 2019-11-29T11:04:59.472 INFO:teuthology.orchestra.run.smithi069:> uname -r 2019-11-29T11:04:59.570 INFO:teuthology.orchestra.run.smithi069.stdout:3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:04:59.570 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:04:59.570 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-11-29T11:04:59.570 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-11-29T11:04:59.570 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.4.3.el7.x86_64"... 2019-11-29T11:04:59.571 INFO:teuthology.orchestra.run.smithi163:> uname -r 2019-11-29T11:04:59.621 INFO:teuthology.orchestra.run.smithi163.stdout:3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:04:59.621 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.4.3.el7.x86_64 2019-11-29T11:04:59.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-11-29T11:05:00.622 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-11-29T11:05:00.623 INFO:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-11-29T11:05:00.759 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2019-11-29T11:05:00.771 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-11-29T11:05:00.771 INFO:teuthology.orchestra.run.smithi163:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-11-29T11:05:00.844 INFO:teuthology.orchestra.run.smithi163.stdout:ttyS1 2019-11-29T11:05:00.861 INFO:teuthology.run_tasks:Running task internal.base... 2019-11-29T11:05:00.871 INFO:teuthology.task.internal:Creating test directory... 2019-11-29T11:05:00.871 INFO:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-11-29T11:05:00.876 INFO:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-11-29T11:05:00.961 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-11-29T11:05:00.972 INFO:teuthology.run_tasks:Running task internal.archive... 2019-11-29T11:05:01.008 INFO:teuthology.task.internal:Creating archive directory... 2019-11-29T11:05:01.009 INFO:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-11-29T11:05:01.014 INFO:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-11-29T11:05:01.103 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-11-29T11:05:01.115 INFO:teuthology.task.internal:Enabling coredump saving... 2019-11-29T11:05:01.115 INFO:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-29T11:05:01.121 INFO:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-29T11:05:01.183 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-29T11:05:01.225 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-29T11:05:01.228 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-11-29T11:05:01.239 INFO:teuthology.task.internal:Configuring sudo... 2019-11-29T11:05:01.240 INFO:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-11-29T11:05:01.244 INFO:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-11-29T11:05:01.348 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-11-29T11:05:01.360 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-11-29T11:05:01.360 INFO:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-11-29T11:05:01.365 INFO:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-11-29T11:05:01.431 INFO:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-29T11:05:01.552 INFO:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-29T11:05:01.680 INFO:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-29T11:05:01.768 INFO:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-29T11:05:01.878 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-11-29T11:05:01.980 INFO:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-29T11:05:02.045 INFO:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-29T11:05:02.116 INFO:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-29T11:05:02.175 INFO:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-29T11:05:02.262 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-11-29T11:05:02.318 INFO:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2019-11-29T11:05:02.323 INFO:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2019-11-29T11:05:02.472 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-11-29T11:05:02.496 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-11-29T11:05:02.527 INFO:teuthology.run_tasks:Running task internal.timer... 2019-11-29T11:05:02.540 INFO:teuthology.task.internal:Starting timer... 2019-11-29T11:05:02.540 INFO:teuthology.run_tasks:Running task pcp... 2019-11-29T11:05:02.576 INFO:teuthology.run_tasks:Running task selinux... 2019-11-29T11:05:02.607 INFO:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2019-11-29T11:05:02.777 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: [ OK ] 2019-11-29T11:05:02.780 INFO:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2019-11-29T11:05:02.868 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: [ OK ] 2019-11-29T11:05:02.870 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-11-29T11:05:02.870 INFO:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2019-11-29T11:05:02.972 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2019-11-29T11:05:02.972 INFO:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2019-11-29T11:05:03.030 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2019-11-29T11:05:03.030 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi163.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive'} 2019-11-29T11:05:03.031 INFO:teuthology.orchestra.run.smithi069:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-11-29T11:05:03.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-29T11:05:03.154 INFO:teuthology.orchestra.run.smithi163:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-11-29T11:05:03.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-29T11:05:03.234 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-11-29T11:05:03.235 INFO:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2019-11-29T11:05:03.359 INFO:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2019-11-29T11:05:03.458 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-11-29T11:05:03.469 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2019-11-29T11:05:03.499 INFO:teuthology.repo_utils:Fetching master from origin 2019-11-29T11:05:03.607 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-11-29T11:05:03.646 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-11-29T11:05:03.651 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-11-29T11:05:04.050 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-11-29T11:05:07.809 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-11-29T11:05:07.834 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:07 +0000 (0:00:00.064) 0:00:00.064 ******* 2019-11-29T11:05:07.848 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:07 +0000 (0:00:00.012) 0:00:00.077 ******* 2019-11-29T11:05:11.228 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi069.front.sepia.ceph.com] 2019-11-29T11:05:11.305 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:11 +0000 (0:00:03.459) 0:00:03.536 ******* 2019-11-29T11:05:11.789 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-11-29T11:05:11.790 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-29T11:05:11.826 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:11 +0000 (0:00:00.521) 0:00:04.058 ******* 2019-11-29T11:05:12.056 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi163.front.sepia.ceph.com] 2019-11-29T11:05:12.133 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.306) 0:00:04.364 ******* 2019-11-29T11:05:12.425 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-11-29T11:05:12.427 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-11-29T11:05:12.463 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.329) 0:00:04.693 ******* 2019-11-29T11:05:12.466 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-11-29T11:05:12.467 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-29T11:05:12.505 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.042) 0:00:04.736 ******* 2019-11-29T11:05:12.510 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-11-29T11:05:12.511 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:12.563 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.058) 0:00:04.794 ******* 2019-11-29T11:05:12.888 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-11-29T11:05:12.890 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-11-29T11:05:12.926 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.362) 0:00:05.157 ******* 2019-11-29T11:05:12.953 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:12.989 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:12 +0000 (0:00:00.062) 0:00:05.220 ******* 2019-11-29T11:05:13.025 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-11-29T11:05:13.026 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:05:13.065 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:13 +0000 (0:00:00.074) 0:00:05.295 ******* 2019-11-29T11:05:13.296 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:05:13.332 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:13 +0000 (0:00:00.268) 0:00:05.563 ******* 2019-11-29T11:05:14.674 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-11-29T11:05:14.674 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025513.08-212837729695655/source", "state": "file", "uid": 0} 2019-11-29T11:05:14.718 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:14 +0000 (0:00:01.385) 0:00:06.949 ******* 2019-11-29T11:05:14.724 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025513.35-3473528554510/source", "state": "file", "uid": 0} 2019-11-29T11:05:14.762 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:14 +0000 (0:00:00.043) 0:00:06.993 ******* 2019-11-29T11:05:15.310 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2019-11-29T11:05:15.310 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:15.342 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:15.625 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:15.660 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:15.908 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:15.950 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.205 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.272 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.543 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.600 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.848 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:16.898 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:17.189 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:17.269 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-29T11:05:17.366 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-11-29T11:05:17.372 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-11-29T11:05:17.557 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:17 +0000 (0:00:02.795) 0:00:09.788 ******* 2019-11-29T11:05:17.591 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:17 +0000 (0:00:00.033) 0:00:09.822 ******* 2019-11-29T11:05:18.562 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi069.front.sepia.ceph.com] 2019-11-29T11:05:18.572 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] 2019-11-29T11:05:18.670 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-11-29T11:05:18.672 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:18 +0000 (0:00:01.079) 0:00:10.902 ******* 2019-11-29T11:05:18.707 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:18 +0000 (0:00:00.036) 0:00:10.938 ******* 2019-11-29T11:05:19.175 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-11-29T11:05:19.176 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.227051", "end": "2019-11-29 11:05:19.153847", "rc": 0, "start": "2019-11-29 11:05:18.926796", "stderr": "", "stderr_lines": [], "stdout": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "stdout_lines": ["4334b3798f2061758a627b6bdd54d0cd6b822faa"]} 2019-11-29T11:05:19.230 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.523) 0:00:11.462 ******* 2019-11-29T11:05:19.237 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.242443", "end": "2019-11-29 11:05:19.174114", "rc": 0, "start": "2019-11-29 11:05:18.931671", "stderr": "", "stderr_lines": [], "stdout": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "stdout_lines": ["4334b3798f2061758a627b6bdd54d0cd6b822faa"]} 2019-11-29T11:05:19.285 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.054) 0:00:11.516 ******* 2019-11-29T11:05:19.569 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-11-29T11:05:19.570 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013650", "end": "2019-11-29 11:05:19.542168", "failed_when_result": false, "rc": 0, "start": "2019-11-29 11:05:19.528518", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-11-29T11:05:19.613 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.325) 0:00:11.841 ******* 2019-11-29T11:05:19.629 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.017897", "end": "2019-11-29 11:05:19.590310", "failed_when_result": false, "rc": 0, "start": "2019-11-29 11:05:19.572413", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-11-29T11:05:19.672 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.057) 0:00:11.898 ******* 2019-11-29T11:05:19.679 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.717 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.049) 0:00:11.948 ******* 2019-11-29T11:05:19.723 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.769 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.051) 0:00:11.999 ******* 2019-11-29T11:05:19.774 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-11-29T11:05:19.776 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.813 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.044) 0:00:12.044 ******* 2019-11-29T11:05:19.819 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.863 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.044) 0:00:12.089 ******* 2019-11-29T11:05:19.864 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-11-29T11:05:19.865 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.907 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.048) 0:00:12.138 ******* 2019-11-29T11:05:19.916 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.956 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.047) 0:00:12.185 ******* 2019-11-29T11:05:19.959 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-11-29T11:05:19.960 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:19.998 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:19 +0000 (0:00:00.044) 0:00:12.229 ******* 2019-11-29T11:05:20.004 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:20.044 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:20 +0000 (0:00:00.045) 0:00:12.275 ******* 2019-11-29T11:05:20.050 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-11-29T11:05:20.051 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:20.089 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:20 +0000 (0:00:00.045) 0:00:12.320 ******* 2019-11-29T11:05:20.098 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:05:20.135 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:20 +0000 (0:00:00.046) 0:00:12.366 ******* 2019-11-29T11:05:20.416 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-11-29T11:05:20.418 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:05:20.455 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:20 +0000 (0:00:00.319) 0:00:12.686 ******* 2019-11-29T11:05:20.484 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:05:20.539 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:05:20 +0000 (0:00:00.084) 0:00:12.771 ******* 2019-11-29T11:05:20.854 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-11-29T11:05:20.928 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-11-29T11:05:21.193 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-11-29T11:05:21.253 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-11-29T11:05:21.523 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-11-29T11:05:21.585 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-11-29T11:05:21.871 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-11-29T11:05:21.873 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-11-29T11:05:22.172 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48 2019-11-29T11:05:22.174 INFO:teuthology.task.ansible.out:cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-11-29T11:05:22.523 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-11-29T11:05:22.525 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-11-29T11:05:22.870 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-11-29T11:05:22.882 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-11-29T11:05:23.184 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-11-29T11:05:23.229 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-11-29T11:05:23.510 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-11-29T11:05:23.543 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-11-29T11:05:23.842 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-11-29T11:05:23.853 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-11-29T11:05:24.165 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-11-29T11:05:24.177 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-11-29T11:05:24.511 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-11-29T11:05:24.533 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-11-29T11:05:24.836 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/sbillah", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "move_home": false, "name": "sbillah", "shell": "/bin/bash", "state": "present", "uid": 1113} 2019-11-29T11:05:24.859 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/sbillah", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "move_home": false, "name": "sbillah", "shell": "/bin/bash", "state": "present", "uid": 1113} 2019-11-29T11:05:25.178 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-11-29T11:05:25.180 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-11-29T11:05:25.447 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-11-29T11:05:25.546 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-11-29T11:05:25.722 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-11-29T11:05:25.883 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-11-29T11:05:25.968 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-11-29T11:05:26.203 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-11-29T11:05:26.320 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-11-29T11:05:26.517 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-11-29T11:05:26.668 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-11-29T11:05:26.875 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-11-29T11:05:26.991 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-11-29T11:05:27.169 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-11-29T11:05:27.338 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-11-29T11:05:27.474 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-11-29T11:05:27.689 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-11-29T11:05:27.805 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-11-29T11:05:28.033 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-11-29T11:05:28.157 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-11-29T11:05:28.370 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-11-29T11:05:28.434 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-11-29T11:05:28.715 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-11-29T11:05:28.769 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-11-29T11:05:29.022 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-11-29T11:05:29.129 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-11-29T11:05:29.326 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-11-29T11:05:29.484 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-11-29T11:05:29.660 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-11-29T11:05:29.796 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-11-29T11:05:30.016 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-11-29T11:05:30.143 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-11-29T11:05:30.361 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-11-29T11:05:30.472 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-11-29T11:05:30.693 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-11-29T11:05:30.820 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-11-29T11:05:31.047 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-11-29T11:05:31.167 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-11-29T11:05:31.374 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-11-29T11:05:31.478 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-11-29T11:05:31.731 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-11-29T11:05:31.808 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-11-29T11:05:32.088 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jj", "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "move_home": false, "name": "jj", "shell": "/bin/bash", "state": "present", "uid": 1135} 2019-11-29T11:05:32.153 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jj", "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "move_home": false, "name": "jj", "shell": "/bin/bash", "state": "present", "uid": 1135} 2019-11-29T11:05:32.460 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-11-29T11:05:32.482 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-11-29T11:05:32.800 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-11-29T11:05:32.822 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-11-29T11:05:33.116 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-11-29T11:05:33.142 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-11-29T11:05:33.456 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-11-29T11:05:33.488 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-11-29T11:05:33.810 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-11-29T11:05:33.828 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-11-29T11:05:34.122 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-11-29T11:05:34.166 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-11-29T11:05:34.468 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-29T11:05:34.480 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-29T11:05:34.755 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-11-29T11:05:34.801 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-11-29T11:05:35.076 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-11-29T11:05:35.172 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-11-29T11:05:35.363 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhishekvrshny", "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "move_home": false, "name": "abhishekvrshny", "shell": "/bin/bash", "state": "present", "uid": 1145} 2019-11-29T11:05:35.526 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhishekvrshny", "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "move_home": false, "name": "abhishekvrshny", "shell": "/bin/bash", "state": "present", "uid": 1145} 2019-11-29T11:05:35.672 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-11-29T11:05:35.856 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-11-29T11:05:35.963 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-11-29T11:05:36.171 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-11-29T11:05:36.267 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-11-29T11:05:36.522 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-11-29T11:05:36.576 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/shehbazj", "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "move_home": false, "name": "shehbazj", "shell": "/bin/bash", "state": "present", "uid": 1149} 2019-11-29T11:05:36.849 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/shehbazj", "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "move_home": false, "name": "shehbazj", "shell": "/bin/bash", "state": "present", "uid": 1149} 2019-11-29T11:05:36.903 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-11-29T11:05:37.181 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-11-29T11:05:37.263 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/fche", "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "move_home": false, "name": "fche", "shell": "/bin/bash", "state": "present", "uid": 1151} 2019-11-29T11:05:37.522 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/fche", "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "move_home": false, "name": "fche", "shell": "/bin/bash", "state": "present", "uid": 1151} 2019-11-29T11:05:37.591 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/onyb", "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "move_home": false, "name": "onyb", "shell": "/bin/bash", "state": "present", "uid": 1152} 2019-11-29T11:05:37.857 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/onyb", "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "move_home": false, "name": "onyb", "shell": "/bin/bash", "state": "present", "uid": 1152} 2019-11-29T11:05:37.943 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-11-29T11:05:38.207 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-11-29T11:05:38.298 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/oprypin", "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "move_home": false, "name": "oprypin", "shell": "/bin/bash", "state": "present", "uid": 1154} 2019-11-29T11:05:38.549 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/oprypin", "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "move_home": false, "name": "oprypin", "shell": "/bin/bash", "state": "present", "uid": 1154} 2019-11-29T11:05:38.613 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-29T11:05:38.874 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-29T11:05:38.943 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-11-29T11:05:39.212 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-11-29T11:05:39.280 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-11-29T11:05:39.540 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-11-29T11:05:39.632 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-11-29T11:05:39.864 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-11-29T11:05:39.966 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-11-29T11:05:40.206 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-11-29T11:05:40.326 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/asheplyakov", "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "move_home": false, "name": "asheplyakov", "shell": "/bin/bash", "state": "present", "uid": 1160} 2019-11-29T11:05:40.552 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/asheplyakov", "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "move_home": false, "name": "asheplyakov", "shell": "/bin/bash", "state": "present", "uid": 1160} 2019-11-29T11:05:40.673 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-11-29T11:05:40.903 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-11-29T11:05:41.019 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-11-29T11:05:41.238 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-11-29T11:05:41.312 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-11-29T11:05:41.576 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-11-29T11:05:41.664 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-11-29T11:05:41.916 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-11-29T11:05:42.015 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-11-29T11:05:42.254 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-11-29T11:05:42.344 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/jwilliamson", "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "move_home": false, "name": "jwilliamson", "shell": "/bin/bash", "state": "present", "uid": 1166} 2019-11-29T11:05:42.593 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/jwilliamson", "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "move_home": false, "name": "jwilliamson", "shell": "/bin/bash", "state": "present", "uid": 1166} 2019-11-29T11:05:42.688 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-11-29T11:05:42.911 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-11-29T11:05:43.018 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-11-29T11:05:43.262 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-11-29T11:05:43.347 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-11-29T11:05:43.618 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-11-29T11:05:43.722 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-11-29T11:05:43.929 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-11-29T11:05:44.068 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-11-29T11:05:44.069 INFO:teuthology.task.ansible.out: 2019-11-29T11:05:44.264 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-11-29T11:05:44.407 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-11-29T11:05:44.591 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-11-29T11:05:44.734 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/liupan", "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "move_home": false, "name": "liupan", "shell": "/bin/bash", "state": "present", "uid": 1173} 2019-11-29T11:05:44.893 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/liupan", "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "move_home": false, "name": "liupan", "shell": "/bin/bash", "state": "present", "uid": 1173} 2019-11-29T11:05:45.076 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-11-29T11:05:45.224 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-11-29T11:05:45.421 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/amarangone", "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "move_home": false, "name": "amarangone", "shell": "/bin/bash", "state": "present", "uid": 1175} 2019-11-29T11:05:45.578 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/amarangone", "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "move_home": false, "name": "amarangone", "shell": "/bin/bash", "state": "present", "uid": 1175} 2019-11-29T11:05:45.768 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-29T11:05:45.859 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-29T11:05:46.110 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/kmroz", "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "move_home": false, "name": "kmroz", "shell": "/bin/bash", "state": "present", "uid": 1176} 2019-11-29T11:05:46.174 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/kmroz", "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "move_home": false, "name": "kmroz", "shell": "/bin/bash", "state": "present", "uid": 1176} 2019-11-29T11:05:46.438 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-11-29T11:05:46.492 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-11-29T11:05:46.770 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-11-29T11:05:46.823 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-11-29T11:05:47.074 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-11-29T11:05:47.173 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-11-29T11:05:47.416 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-11-29T11:05:47.494 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-11-29T11:05:47.733 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-11-29T11:05:47.849 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-11-29T11:05:48.037 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-11-29T11:05:48.217 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-11-29T11:05:48.385 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/adamyanova", "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "move_home": false, "name": "adamyanova", "shell": "/bin/bash", "state": "present", "uid": 1183} 2019-11-29T11:05:48.574 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/adamyanova", "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "move_home": false, "name": "adamyanova", "shell": "/bin/bash", "state": "present", "uid": 1183} 2019-11-29T11:05:48.715 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-11-29T11:05:48.925 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-11-29T11:05:49.070 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-11-29T11:05:49.271 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-11-29T11:05:49.411 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-11-29T11:05:49.632 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-11-29T11:05:49.761 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-11-29T11:05:49.959 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-11-29T11:05:50.058 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-11-29T11:05:50.271 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-11-29T11:05:50.443 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-11-29T11:05:50.602 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-11-29T11:05:50.780 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-11-29T11:05:50.949 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-11-29T11:05:51.098 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-11-29T11:05:51.277 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-11-29T11:05:51.435 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-11-29T11:05:51.614 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-11-29T11:05:51.754 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-11-29T11:05:51.935 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-11-29T11:05:52.095 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-11-29T11:05:52.284 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-11-29T11:05:52.404 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-11-29T11:05:52.616 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-11-29T11:05:52.712 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-11-29T11:05:52.966 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-11-29T11:05:53.051 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-11-29T11:05:53.306 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-11-29T11:05:53.336 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-11-29T11:05:53.651 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-11-29T11:05:53.683 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-11-29T11:05:53.999 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-11-29T11:05:54.011 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-11-29T11:05:54.308 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-11-29T11:05:54.340 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-11-29T11:05:54.671 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-11-29T11:05:54.680 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-11-29T11:05:55.002 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-11-29T11:05:55.034 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-11-29T11:05:55.363 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-11-29T11:05:55.386 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-11-29T11:05:55.687 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-11-29T11:05:55.721 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-11-29T11:05:56.027 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-11-29T11:05:56.030 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-11-29T11:05:56.351 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-11-29T11:05:56.373 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-11-29T11:05:56.645 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-11-29T11:05:56.688 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-11-29T11:05:56.959 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-11-29T11:05:57.023 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-11-29T11:05:57.314 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-11-29T11:05:57.367 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-11-29T11:05:57.670 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-11-29T11:05:57.703 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-11-29T11:05:57.980 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-11-29T11:05:58.012 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-11-29T11:05:58.305 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-11-29T11:05:58.317 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-11-29T11:05:58.643 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-11-29T11:05:58.665 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-11-29T11:05:58.998 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-11-29T11:05:58.999 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-29T11:05:59.318 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-11-29T11:05:59.340 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-29T11:05:59.670 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-11-29T11:06:00.002 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-11-29T11:06:00.255 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:39.715) 0:00:52.486 ******* 2019-11-29T11:06:00.282 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-11-29T11:06:00.325 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:00.070) 0:00:52.557 ******* 2019-11-29T11:06:00.375 INFO:teuthology.task.ansible.out: 2019-11-29T11:06:00.376 INFO:teuthology.task.ansible.out:TASK [users : Merge managed_users and managed_admin users] ********************* 2019-11-29T11:06:00.379 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-11-29T11:06:00.380 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04 2019-11-29T11:06:00.382 INFO:teuthology.task.ansible.out:okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "jos 2019-11-29T11:06:00.383 INFO:teuthology.task.ansible.out:hd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jo 2019-11-29T11:06:00.384 INFO:teuthology.task.ansible.out:wilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f 2019-11-29T11:06:00.386 INFO:teuthology.task.ansible.out:966b2794c94caa110d313e927a1c1b\n"}, {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h 2019-11-29T11:06:00.387 INFO:teuthology.task.ansible.out:7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0 2019-11-29T11:06:00.388 INFO:teuthology.task.ansible.out:bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, {"name": "gabrioux 2019-11-29T11:06:00.390 INFO:teuthology.task.ansible.out:", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "o 2019-11-29T11:06:00.391 INFO:teuthology.task.ansible.out:vpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e 2019-11-29T11:06:00.392 INFO:teuthology.task.ansible.out:5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, 2019-11-29T11:06:00.394 INFO:teuthology.task.ansible.out: {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-11-29T11:06:00.447 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:00.121) 0:00:52.678 ******* 2019-11-29T11:06:00.453 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-11-29T11:06:00.453 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-11-29T11:06:00.600 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:00.153) 0:00:52.832 ******* 2019-11-29T11:06:00.628 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-11-29T11:06:00.667 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:00.066) 0:00:52.898 ******* 2019-11-29T11:06:00.728 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-11-29T11:06:00.733 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-11-29T11:06:00.734 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04 2019-11-29T11:06:00.736 INFO:teuthology.task.ansible.out:okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "jos 2019-11-29T11:06:00.737 INFO:teuthology.task.ansible.out:hd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jo 2019-11-29T11:06:00.738 INFO:teuthology.task.ansible.out:wilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f 2019-11-29T11:06:00.740 INFO:teuthology.task.ansible.out:966b2794c94caa110d313e927a1c1b\n"}, {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h 2019-11-29T11:06:00.741 INFO:teuthology.task.ansible.out:7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6 2019-11-29T11:06:00.743 INFO:teuthology.task.ansible.out:c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa 2019-11-29T11:06:00.744 INFO:teuthology.task.ansible.out:2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db2 2019-11-29T11:06:00.746 INFO:teuthology.task.ansible.out:93dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.red 2019-11-29T11:06:00.747 INFO:teuthology.task.ansible.out:hat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f 2019-11-29T11:06:00.748 INFO:teuthology.task.ansible.out:57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-11-29T11:06:00.790 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:00 +0000 (0:00:00.123) 0:00:53.022 ******* 2019-11-29T11:06:01.410 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi163.front.sepia.ceph.com -> localhost] => {"after": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "attempts": 1, "before": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "changed": false, "remote_url_changed": false} 2019-11-29T11:06:01.450 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:01 +0000 (0:00:00.659) 0:00:53.681 ******* 2019-11-29T11:06:01.478 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com -> localhost] => {"after": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "attempts": 1, "before": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "changed": false, "remote_url_changed": false} 2019-11-29T11:06:01.516 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:01 +0000 (0:00:00.066) 0:00:53.747 ******* 2019-11-29T11:06:01.876 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-11-29T11:06:01.922 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-11-29T11:06:02.209 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-11-29T11:06:02.276 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-11-29T11:06:02.517 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-11-29T11:06:02.633 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-11-29T11:06:02.856 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-11-29T11:06:02.988 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-11-29T11:06:03.217 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-11-29T11:06:03.218 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-11-29T11:06:03.322 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-11-29T11:06:03.326 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-11-29T11:06:03.554 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-11-29T11:06:03.659 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-11-29T11:06:03.879 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-11-29T11:06:03.881 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-11-29T11:06:04.013 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-11-29T11:06:04.015 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-11-29T11:06:04.211 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-11-29T11:06:04.359 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-11-29T11:06:04.548 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-11-29T11:06:04.558 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:04.696 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-11-29T11:06:04.706 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:04.906 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-11-29T11:06:04.907 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-11-29T11:06:04.927 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:04.939 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:04.939 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:04.952 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:05.038 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-11-29T11:06:05.039 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-11-29T11:06:05.054 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/W 2019-11-29T11:06:05.060 INFO:teuthology.task.ansible.out:ZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:05.073 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:05.297 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-11-29T11:06:05.411 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-11-29T11:06:05.678 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-11-29T11:06:05.679 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-11-29T11:06:05.762 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-11-29T11:06:05.764 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-11-29T11:06:06.035 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-11-29T11:06:06.132 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-11-29T11:06:06.381 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-11-29T11:06:06.469 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-11-29T11:06:06.746 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-11-29T11:06:06.841 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-11-29T11:06:07.105 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-11-29T11:06:07.179 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-11-29T11:06:07.473 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-11-29T11:06:07.536 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-11-29T11:06:07.822 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-11-29T11:06:07.865 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-11-29T11:06:08.136 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-11-29T11:06:08.138 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-11-29T11:06:08.221 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-11-29T11:06:08.222 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-11-29T11:06:08.500 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-11-29T11:06:08.506 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-11-29T11:06:08.563 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-11-29T11:06:08.565 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-11-29T11:06:08.842 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-11-29T11:06:08.884 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-11-29T11:06:09.216 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-11-29T11:06:09.254 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-11-29T11:06:09.553 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-11-29T11:06:09.554 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-11-29T11:06:09.596 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-11-29T11:06:09.598 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-11-29T11:06:09.859 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-11-29T11:06:09.954 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-11-29T11:06:10.216 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-11-29T11:06:10.322 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-11-29T11:06:10.574 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-11-29T11:06:10.680 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-11-29T11:06:10.947 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-11-29T11:06:11.020 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-11-29T11:06:11.294 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-11-29T11:06:11.358 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-11-29T11:06:11.646 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-11-29T11:06:11.647 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-11-29T11:06:11.722 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-11-29T11:06:11.723 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-11-29T11:06:11.995 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXvd7F0n4DeBROTLYDE4lItYn/2S1cMM1uAkkwissUh4LGsMAVWi5tUI0EUC7I2BUxACvIIA4yfkhe2ATAu95UQxKWVql67sfCW6WwY7lNa65kQC9S5dE/+d4vPxFoFSFnri981laJXOjbjTsWD7j1SGfqbRuAHQKL8y95DQ3y/B7/1oxTg1jjK8BTr3GAUp8/7lNlWE+x2S3gjHqqH92yWU7OBbPWRZjjkVkXS/5CbWDDUx8nbeGPcAIfM95rRQR0VkwzVi1FWsXd6qIbZ9TqOhYyLR0fCsnDuveNV68FB1ddvqCLdXRxPLmucwzdjGMbZ5EJ0H/CAAutj3Vjn6gh jj@newdream.net\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRXOkus/WHB3llEH1u4rWj0F/bZSslq4/o7pBUGFYrHKqa2iBiCj4DxBEsaReWUm0aDyesGuEVeoECTtzh7byJ6iZHFbf0McubxnuFMImneQ2mB0+raa3uWblwnyiwAqiZgtcJGQpCxCnrmLzyNN6xUUmzN27cDOMSDpbWr3VdPSBH+ixIuZI8r8UkSbmlX30V2bvk4lGl2SPV385tAvRPRU9KfMG9AgwNv+R2WCiA2GOrjes2VHm7LneqI/vE5yOxJsm4tPxEkOpCp3/0xt9L9cmPz47HnzJ2Qure3kZmOsCxOX1+9dXFIwhPaNconD4KDlv4MNs/M425Gp6KKnnV jj@laptop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBjrQ3drodiPr83uvpUUf6SXZ0utOFIfy5Cj3BGcn7bY29BKXCwomAUmmLMk76cGBZ7jHf3ZJYid1ESbHmvmSgKFShr9JFXBHvlMdnyewi7hFix+sjX9ZtuZfSaIsBbU5Jlz3RqHtDrScS1wJRh+ 2019-11-29T11:06:11.997 INFO:teuthology.task.ansible.out:oDlH6is5QWSCABVWhUk+hJtQFg4dfespqomt88NmPZuR0aR0b22hkrMYINXP0y3iz3Jjs5vg8td7VJ2hXb5yIBk+Izuq1LnjPyRCZv5LkZMuwofexpfflLKeVHw0t31HXuzBUZMoKqXiGJweyPDzY/xbeTVnYvQQEvoGxRDpaYuiGSfhFW0UHE3HSeuYfklO2x juanjose@aurora", "key_options": null, "keyfile": "/home/jj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jj", "validate_certs": true} 2019-11-29T11:06:12.093 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXvd7F0n4DeBROTLYDE4lItYn/2S1cMM1uAkkwissUh4LGsMAVWi5tUI0EUC7I2BUxACvIIA4yfkhe2ATAu95UQxKWVql67sfCW6WwY7lNa65kQC9S5dE/+d4vPxFoFSFnri981laJXOjbjTsWD7j1SGfqbRuAHQKL8y95DQ3y/B7/1oxTg1jjK8BTr3GAUp8/7lNlWE+x2S3gjHqqH92yWU7OBbPWRZjjkVkXS/5CbWDDUx8nbeGPcAIfM95rRQR0VkwzVi1FWsXd6qIbZ9TqOhYyLR0fCsnDuveNV68FB1ddvqCLdXRxPLmucwzdjGMbZ5EJ0H/CAAutj3Vjn6gh jj@newdream.net\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRXOkus/WHB3llEH1u4rWj0F/bZSslq4/o7pBUGFYrHKqa2iBiCj4DxBEsaReWUm0aDyesGuEVeoECTtzh7byJ6iZHFbf0McubxnuFMImneQ2mB0+raa3uWblwnyiwAqiZgtcJGQpCxCnrmLzyNN6xUUmzN27cDOMSDpbWr3VdPSBH+ixIuZI8r8UkSbmlX30V2bvk4lGl2SPV385tAvRPRU9KfMG9AgwNv+R2WCiA2GOrjes2VHm7LneqI/vE5yOxJsm4tPxEkOpCp3/0xt9L9cmPz47HnzJ2Qure3kZmOsCxOX1+9dXFIwhPaNconD4KDlv4MNs/M425Gp6KKnnV jj@laptop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBjrQ3drodiPr83uvpUUf6SXZ0utOFIfy5Cj3BGcn7bY29BKXCwomAUmmLMk76cGBZ7jHf3ZJYid1ESbHmvmSgKFShr9JFXBHvlMdnyewi7hFix+sjX9ZtuZfSaIsBbU5Jlz3RqHtDrScS1wJRh+ 2019-11-29T11:06:12.094 INFO:teuthology.task.ansible.out:oDlH6is5QWSCABVWhUk+hJtQFg4dfespqomt88NmPZuR0aR0b22hkrMYINXP0y3iz3Jjs5vg8td7VJ2hXb5yIBk+Izuq1LnjPyRCZv5LkZMuwofexpfflLKeVHw0t31HXuzBUZMoKqXiGJweyPDzY/xbeTVnYvQQEvoGxRDpaYuiGSfhFW0UHE3HSeuYfklO2x juanjose@aurora", "key_options": null, "keyfile": "/home/jj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jj", "validate_certs": true} 2019-11-29T11:06:12.304 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-11-29T11:06:12.431 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-11-29T11:06:12.620 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-11-29T11:06:12.790 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-11-29T11:06:12.936 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-11-29T11:06:13.133 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-11-29T11:06:13.258 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-11-29T11:06:13.488 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-11-29T11:06:13.595 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-11-29T11:06:13.830 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-11-29T11:06:13.884 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-11-29T11:06:14.149 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-11-29T11:06:14.182 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-29T11:06:14.501 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-29T11:06:14.512 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-11-29T11:06:14.839 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-11-29T11:06:14.850 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-11-29T11:06:15.169 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAfEhx/xZpJV9ouKFW4QBC46cNicIQMzU8DSYl1GOP9+pnnD+AudVQubvrEePGA7TFkSar5Ccv4dnoQ2s1CaPAtbT+s+nQlf5OFO9P/qvhkM4JhlmH2XxgJAcOvZH2W1HXfGTc4WIuU/OtXM+yWMGlTXxqMqwCvGfqc+Yf8zkt0t3m0kBdWoFcZvdYqKuS15bvdVuiKz7GUxDrtw98yMXb/ob/E+Ygmmphn9dMatTeZ6SD1fbcrHh9oIVbYiY93+++jXzjEXrGzJh/gphgEtvIf8UPMF9HiXXPqwhxneRvghvZ/B05LGfcsqGAtyK09eF94hglyl9kJIdm/d7lmmM/ abhishek.varshney@VS-C02PJ2CSFVH5.local", "key_options": null, "keyfile": "/home/abhishekvrshny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhishekvrshny", "validate_certs": true} 2019-11-29T11:06:15.195 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-11-29T11:06:15.484 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-11-29T11:06:15.487 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-11-29T11:06:15.527 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAfEhx/xZpJV9ouKFW4QBC46cNicIQMzU8DSYl1GOP9+pnnD+AudVQubvrEePGA7TFkSar5Ccv4dnoQ2s1CaPAtbT+s+nQlf5OFO9P/qvhkM4JhlmH2XxgJAcOvZH2W1HXfGTc4WIuU/OtXM+yWMGlTXxqMqwCvGfqc+Yf8zkt0t3m0kBdWoFcZvdYqKuS15bvdVuiKz7GUxDrtw98yMXb/ob/E+Ygmmphn9dMatTeZ6SD1fbcrHh9oIVbYiY93+++jXzjEXrGzJh/gphgEtvIf8UPMF9HiXXPqwhxneRvghvZ/B05LGfcsqGAtyK09eF94hglyl9kJIdm/d7lmmM/ abhishek.varshney@VS-C02PJ2CSFVH5.local", "key_options": null, "keyfile": "/home/abhishekvrshny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhishekvrshny", "validate_certs": true} 2019-11-29T11:06:15.809 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-11-29T11:06:15.815 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZ 2019-11-29T11:06:15.835 INFO:teuthology.task.ansible.out:b9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-11-29T11:06:16.126 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-11-29T11:06:16.153 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-11-29T11:06:16.168 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-11-29T11:06:16.455 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUqFGbkDz4L4BpIs2znPSWeawZZM2FBLrR3htaMBM6lcmiX2h+vgVGv2ZLpajGo7KB6J59kzWLBCJXN2DG0uUqa7dOdjEDuDrjZkNgDzXZ20lqzFAjLwNF6a9PcDN/oX6jzK9+3xArleQuTIJsOPRue2dQYGRnyIIrYpapJkkUgI+lXfwrEixCY3xjGNvVoh5bqQmkk3buLIvKEMwhPXJ94SGqsNTNC75HfKhVMjxA8QmuiXJVEgiW00V3Cctwv+yjiDMts4Dcs6yEzxXzjv0c0APxxN78bKlkwFgXiqawLiApk752/x2teSg7kvdrlGcCAWq/BhwphYjwaRlXIj1N shehbaz@donjaffer", "key_options": null, "keyfile": "/home/shehbazj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shehbazj", "validate_certs": true} ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-11-29T11:06:16.457 INFO:teuthology.task.ansible.out: 2019-11-29T11:06:16.772 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUqFGbkDz4L4BpIs2znPSWeawZZM2FBLrR3htaMBM6lcmiX2h+vgVGv2ZLpajGo7KB6J59kzWLBCJXN2DG0uUqa7dOdjEDuDrjZkNgDzXZ20lqzFAjLwNF6a9PcDN/oX6jzK9+3xArleQuTIJsOPRue2dQYGRnyIIrYpapJkkUgI+lXfwrEixCY3xjGNvVoh5bqQmkk3buLIvKEMwhPXJ94SGqsNTNC75HfKhVMjxA8QmuiXJVEgiW00V3Cctwv+yjiDMts4Dcs6yEzxXzjv0c0APxxN78bKlkwFgXiqawLiApk752/x2teSg7kvdrlGcCAWq/BhwphYjwaRlXIj1N shehbaz@donjaffer", "key_options": null, "keyfile": "/home/shehbazj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shehbazj", "validate_certs": true} 2019-11-29T11:06:16.804 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-11-29T11:06:17.119 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-11-29T11:06:17.173 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJorrIT68EZc7A4IHwQGv8uoblSAIUnMxhOq6rlnvmqx4clpUzKkbzzrZUctP47aeqaSX5Mc6mV8TY0sIH8fFQwahF3ueTKT3YecZvPwiK4ALmawkvFpmgnJKKYhTrEApbFBhtH0KnNLtozpl4RJds4p09maNeoUXJDy3A24tpXViU2Nrw8lSIT6VJzvqSlK+miNEf3VKI6gnRB7fGcJe913zITf09Rew7GexAyQUYHusZbJCf5rl6hkewkZMkgbXfG22f/XWcYjoRtA/ZqeWIP2UDO/H3tfJnlMxUshbDGLllMFJzwmwhkYbapqVmnu7Y01rEQ4LqvaHZMr/yKn+N fche@very.elastic.org", "key_options": null, "keyfile": "/home/fche/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "fche", "validate_certs": true} 2019-11-29T11:06:17.469 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJorrIT68EZc7A4IHwQGv8uoblSAIUnMxhOq6rlnvmqx4clpUzKkbzzrZUctP47aeqaSX5Mc6mV8TY0sIH8fFQwahF3ueTKT3YecZvPwiK4ALmawkvFpmgnJKKYhTrEApbFBhtH0KnNLtozpl4RJds4p09maNeoUXJDy3A24tpXViU2Nrw8lSIT6VJzvqSlK+miNEf3VKI6gnRB7fGcJe913zITf09Rew7GexAyQUYHusZbJCf5rl6hkewkZMkgbXfG22f/XWcYjoRtA/ZqeWIP2UDO/H3tfJnlMxUshbDGLllMFJzwmwhkYbapqVmnu7Y01rEQ4LqvaHZMr/yKn+N fche@very.elastic.org", "key_options": null, "keyfile": "/home/fche/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "fche", "validate_certs": true} 2019-11-29T11:06:17.511 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmKchKXS6xKpo+25ZN5vMpbuL4OoFZWPfPSxur6iaBKdr6LSMejnAwkJOhfhH55pZLuAglHWdMcZi8H/ktzsZVd+ybvJNhpZsXdefNqqxUyjn320cT73pDdDC/W1519fUiokss23S6B4GR0B44VrVKezN5Dyhf4sATN8l76QnIALVK5hvYOI3v5k70weaPJM4fma7xYF5b6qSxauGofzMb5XwjNaaPE9F8U08nxGk5XK+S0rHoFnCjoKBlN9Ee1gk6PjsStQAyK5ujWvPGtG/UvuU9LewVbt6wuPbpwsf7Ck0u4wHECHK+185JF/X+qsWwv030pKShXblLVckNVhAx ani@orchid", "key_options": null, "keyfile": "/home/onyb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "onyb", "validate_certs": true} 2019-11-29T11:06:17.822 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmKchKXS6xKpo+25ZN5vMpbuL4OoFZWPfPSxur6iaBKdr6LSMejnAwkJOhfhH55pZLuAglHWdMcZi8H/ktzsZVd+ybvJNhpZsXdefNqqxUyjn320cT73pDdDC/W1519fUiokss23S6B4GR0B44VrVKezN5Dyhf4sATN8l76QnIALVK5hvYOI3v5k70weaPJM4fma7xYF5b6qSxauGofzMb5XwjNaaPE9F8U08nxGk5XK+S0rHoFnCjoKBlN9Ee1gk6PjsStQAyK5ujWvPGtG/UvuU9LewVbt6wuPbpwsf7Ck0u4wHECHK+185JF/X+qsWwv030pKShXblLVckNVhAx ani@orchid", "key_options": null, "keyfile": "/home/onyb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "onyb", "validate_certs": true} ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-11-29T11:06:18.163 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-11-29T11:06:18.212 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzf8fjotuelTIdz7clDpSN/VDtXirYrmzM/r157Np9uTMgn/rhzzplp2DKqFrCEDwawJ1O1Fr/H5DqttPonyRmjPmAIf96P8KKLsC600R63xk4DNhnByY0WBU3o1e46LP7DJLsOOy6hZRbyjgoFfzcapH7aV+fa37k7/Ri5mpmdmENa6lYI38+YaItFJf0kTXdFou0iLAt7xogOTimgblMst6UjOhRIsMFA3QS4PvOFR+AedCHeVtYK9mAeRtQ530wKzjz8x4NFU3Y93jmygwTTG2RTEfRjfbObfFOQXJGKcl28pxGPUL9ZlrLx1juAekn2Uz+2oO/eQ8G5w+XW3rp", "key_options": null, "keyfile": "/home/oprypin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oprypin", "validate_certs": true} 2019-11-29T11:06:18.511 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzf8fjotuelTIdz7clDpSN/VDtXirYrmzM/r157Np9uTMgn/rhzzplp2DKqFrCEDwawJ1O1Fr/H5DqttPonyRmjPmAIf96P8KKLsC600R63xk4DNhnByY0WBU3o1e46LP7DJLsOOy6hZRbyjgoFfzcapH7aV+fa37k7/Ri5mpmdmENa6lYI38+YaItFJf0kTXdFou0iLAt7xogOTimgblMst6UjOhRIsMFA3QS4PvOFR+AedCHeVtYK9mAeRtQ530wKzjz8x4NFU3Y93jmygwTTG2RTEfRjfbObfFOQXJGKcl28pxGPUL9ZlrLx1juAekn2Uz+2oO/eQ8G5w+XW3rp", "key_options": null, "keyfile": "/home/oprypin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oprypin", "validate_certs": true} 2019-11-29T11:06:18.555 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-11-29T11:06:18.557 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-29T11:06:18.875 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-11-29T11:06:18.876 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-29T11:06:18.904 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-11-29T11:06:18.906 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-11-29T11:06:18.908 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-11-29T11:06:19.232 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-11-29T11:06:19.233 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-11-29T11:06:19.235 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-11-29T11:06:19.243 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-11-29T11:06:19.569 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-11-29T11:06:19.581 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-11-29T11:06:19.908 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-11-29T11:06:19.928 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-11-29T11:06:20.268 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDLCJHta/5O2C2MXRegwo+bJUiqbkESr+RwoH/VIdb4L0JDlVsqV+K+BOBjLZy3vwzWCaSackvshoZHAwGvvfGXjtLbhAYs6QWWHODf/XWBvOW4a96fbTToNpV1dXkHAGqlJMYln3GKVD0o9b69iQUTcLILJhWQjCoTjZsvTtP9OYn9YLfuoxhaaU3PazZVhZYml4Ca/3ns9vNQ/m+v4nh82ysyflGKYozEq0pl8VH0Ydy/FtOO98PhQx3crrivu3qghubntVRQnULUmlDSFhMff+ukLVdj2U0AdSBatlJuaSofWUMAQkv0Ia/om9doDGv2TqkXhJnZdGaWJwmuh6rKIZr7d7TYTD6Uag3wJ1NKiDETRNtmMCLuniEJzPO4asBpSSWGyVatyrNmAgBb6XCeZRkRcYvTXdA5OESKG3HJeguk0XCrjosIEYcvxyye9SgA8U73jWdDUdh0yIHLWcfrgC2eynE1uBnTPNBvtnf9pKqMEp5opdRAUhckhPRxJQ2JHcYMAFnq7gz1bL2boL2ElZOqHBx9qgv0ZKnk+YXqhyp2EP2JK0urd0EHMqkGUsRc7D0FSrz+j4lSkd71sHu3tMgwIrgDR55wYMBBRXTrR8pGTMnGldDBaxvUCgWnlsTQJdjs9xCM5gcJzzgZgkRBmrTOUkw2ABre6KMRAmQ4Zg3HxO0MBr5rfV5glWXuDJXsEowLPp/NQ+3X7PIU9LAtKkcZPWC3MayoFdEdNpKieCgOfRAiwJ34fWz4ozM0IvENmUmCKz8+yDAOeupJJ1/367Rl3+U8XTsDNSsqcmsRdr8ZnUIN0662FGr/nHkVaDq79rEywt/V2eMLWyunF0FUUnbwBB8/J0hgIQPX1ncziuzPTF/YCTR6lgVEtvNHPI1xTQsaVQkF9Whu9Eq6Gz1cFq6H/abb1DLJAoO1Cy0HjoT8MjwOICVcsrqxGXsNRBsv4fCxOaz6+WyVWz2O5gpNKOLLmenr9JbiTGjamjfl9dDempRROSDjsJJY6TPdRb0WBuk3i6mOAsAYeBHoargi+a+dQ1IDzGdVY2mUXtDTA6HluZVvzEKnAR6QzuEdRLOrbteETRQ3cV9STXgscCTokzDNzhYbkLUIl/SRBYBdPMiR9TgW+m/0IJkBTEEUuDtYVzyHezV94F17vzpK+xbRdk3/WmWPScPDkc9tDp0AGLdZ/FAkNzGM1VE8G5OJ7OduogIdIQW6yR8y+CP1O4e2WYZ3M0IjJFX8pc/zBinUBNOBvgAzO8aSXTgxXbs/Q1fcv7htmJZ/3nnB4U6N85tq5oklCWulk3tIFI5pMGiQUoQCW8PH2nwasaKKtOInBZMWm8B2K0c5bU2/bKRcXJNJ asheplyakov@as-ubuntu", "key_options": null, "keyfile": "/home/asheplyakov/.ssh/authorized_keys 2019-11-29T11:06:20.270 INFO:teuthology.task.ansible.out:", "manage_dir": true, "path": null, "state": "present", "user": "asheplyakov", "validate_certs": true} 2019-11-29T11:06:20.281 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-11-29T11:06:20.624 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-11-29T11:06:20.635 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDLCJHta/5O2C2MXRegwo+bJUiqbkESr+RwoH/VIdb4L0JDlVsqV+K+BOBjLZy3vwzWCaSackvshoZHAwGvvfGXjtLbhAYs6QWWHODf/XWBvOW4a96fbTToNpV1dXkHAGqlJMYln3GKVD0o9b69iQUTcLILJhWQjCoTjZsvTtP9OYn9YLfuoxhaaU3PazZVhZYml4Ca/3ns9vNQ/m+v4nh82ysyflGKYozEq0pl8VH0Ydy/FtOO98PhQx3crrivu3qghubntVRQnULUmlDSFhMff+ukLVdj2U0AdSBatlJuaSofWUMAQkv0Ia/om9doDGv2TqkXhJnZdGaWJwmuh6rKIZr7d7TYTD6Uag3wJ1NKiDETRNtmMCLuniEJzPO4asBpSSWGyVatyrNmAgBb6XCeZRkRcYvTXdA5OESKG3HJeguk0XCrjosIEYcvxyye9SgA8U73jWdDUdh0yIHLWcfrgC2eynE1uBnTPNBvtnf9pKqMEp5opdRAUhckhPRxJQ2JHcYMAFnq7gz1bL2boL2ElZOqHBx9qgv0ZKnk+YXqhyp2EP2JK0urd0EHMqkGUsRc7D0FSrz+j4lSkd71sHu3tMgwIrgDR55wYMBBRXTrR8pGTMnGldDBaxvUCgWnlsTQJdjs9xCM5gcJzzgZgkRBmrTOUkw2ABre6KMRAmQ4Zg3HxO0MBr5rfV5glWXuDJXsEowLPp/NQ+3X7PIU9LAtKkcZPWC3MayoFdEdNpKieCgOfRAiwJ34fWz4ozM0IvENmUmCKz8+yDAOeupJJ1/367Rl3+U8XTsDNSsqcmsRdr8ZnUIN0662FGr/nHkVaDq79rEywt/V2eMLWyunF0FUUnbwBB8/J0hgIQPX1ncziuzPTF/YCTR6lgVEtvNHPI1xTQsaVQkF9Whu9Eq6Gz1cFq6H/abb1DLJAoO1Cy0HjoT8MjwOICVcsrqxGXsNRBsv4fCxOaz6+WyVWz2O5gpNKOLLmenr9JbiTGjamjfl9dDempRROSDjsJJY6TPdRb0WBuk3i6mOAsAYeBHoargi+a+dQ1IDzGdVY2mUXtDTA6HluZVvzEKnAR6QzuEdRLOrbteETRQ3cV9STXgscCTokzDNzhYbkLUIl/SRBYBdPMiR9TgW+m/0IJkBTEEUuDtYVzyHezV94F17vzpK+xbRdk3/WmWPScPDkc9tDp0AGLdZ/FAkNzGM1VE8G5OJ7OduogIdIQW6yR8y+CP1O4e2WYZ3M0IjJFX8pc/zBinUBNOBvgAzO8aSXTgxXbs/Q1fcv7htmJZ/3nnB4U6N85tq5oklCWulk3tIFI5pMGiQUoQCW8PH2nwasaKKtOInBZMWm8B2K0c5bU2/bKRcXJNJ asheplyakov@as-ubuntu", "key_options": null, "keyfile": "/home/asheplyakov/.ssh/authorized_keys 2019-11-29T11:06:20.637 INFO:teuthology.task.ansible.out:", "manage_dir": true, "path": null, "state": "present", "user": "asheplyakov", "validate_certs": true} 2019-11-29T11:06:20.918 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-11-29T11:06:21.002 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-11-29T11:06:21.244 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-11-29T11:06:21.319 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-11-29T11:06:21.550 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-11-29T11:06:21.666 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-11-29T11:06:21.896 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-11-29T11:06:21.992 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-11-29T11:06:22.240 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCmp/WxBoQxK/3bI3N9X4INmChIFtG/75NULYt9CAAnoOhFHe8xHMOU4x/YS6JdoIBJXusrjbDhCksFSK/caoOow9CQp3VWytslRSjfjkjbwwWHSdcyGpSO4lS5F32SgLOZBuyktpkLDgi6Ou5OTNKokJO8qiZo9Uw0rEBNakfsVi+fpysUDL792Jk1ROUr0HUpwqrQfTPZaue+PomXPQSYIqf06EH1nJYho6D4jt/g6U0qc66WTW5dXTkWD2VVQjZbFGkamz0ICHBJ/Xpf4TjWQVotsYD5NtbvqPo6ygpJYNhKR2n2zA+L4Zu2n0AEib5S4ftNxc7gNDCCfVs/EEIvXp/3CjKVPxHf4xUUPUZj+SQFnSUPQfeY1iIydoX+IG9oj5kKmQBdQFxbAaZJ679bORIqlH73dUFGr2rfS1UlIgPW5ll77bixsB0GP3+Jw4FF5QbO+k1d5qJpUw+8D6FbWAppP7hnSAok9PTpBqfkMpFiX00ZTIFwKCBLxdtJOfbIhbOPCnyEoB2ZLfRiBJmXhBpLjGghWIPPnGZh+yeJiy/EyKwN57vLmjdmvcDgSsjHswT+rQ+QzPoF+/KcdU1LL6ME6i+R7o7aiLOdDpZo4p5d1kr3VMNjVNM+L9ECpTntPhyrglOX018Q8xcNRh2/z+G6NsiiV4t0YxFdk/ICTQ== Jesse.Williamson@suse.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXsBAd8JfyMT352SjfhZ2Nj3gR2PkeyPS9yB6fKck5CW0Ns1EcUAOIFYw7a1R6zCDJbkkXZDRgM1W6v/UBgpOm69HaNCZuqx19mQRNgo1lhw6hDHc+9754wOmIbgu3gInIS/XPvGzAkGwBW3rWlOaNurY4Y/PH1QyiaujD1Ii9X2Q6yRP8hsDrccual36gngY6wV3G4xKbS8kTY4qQekHkgFT+ZDHv9ibliDfAxtzm5X8N25sICpvcnQkG+Co3QP2XADEhoQrs8L7QekKYPlX8zq6rJBO0cGYW50E58LJk7s67n7QQPCy04xeAf9KOZZBSMHTI8eSFHjb8B034IkHH jwilliamson@linux-z0c6", "key_options": null, "keyfile": "/home/jwilliamson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jwilliamson", "validate_certs": true} 2019-11-29T11:06:22.304 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-11-29T11:06:22.535 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-11-29T11:06:22.536 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-11-29T11:06:22.652 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCmp/WxBoQxK/3bI3N9X4INmChIFtG/75NULYt9CAAnoOhFHe8xHMOU4x/YS6JdoIBJXusrjbDhCksFSK/caoOow9CQp3VWytslRSjfjkjbwwWHSdcyGpSO4lS5F32SgLOZBuyktpkLDgi6Ou5OTNKokJO8qiZo9Uw0rEBNakfsVi+fpysUDL792Jk1ROUr0HUpwqrQfTPZaue+PomXPQSYIqf06EH1nJYho6D4jt/g6U0qc66WTW5dXTkWD2VVQjZbFGkamz0ICHBJ/Xpf4TjWQVotsYD5NtbvqPo6ygpJYNhKR2n2zA+L4Zu2n0AEib5S4ftNxc7gNDCCfVs/EEIvXp/3CjKVPxHf4xUUPUZj+SQFnSUPQfeY1iIydoX+IG9oj5kKmQBdQFxbAaZJ679bORIqlH73dUFGr2rfS1UlIgPW5ll77bixsB0GP3+Jw4FF5QbO+k1d5qJpUw+8D6FbWAppP7hnSAok9PTpBqfkMpFiX00ZTIFwKCBLxdtJOfbIhbOPCnyEoB2ZLfRiBJmXhBpLjGghWIPPnGZh+yeJiy/EyKwN57vLmjdmvcDgSsjHswT+rQ+QzPoF+/KcdU1LL6ME6i+R7o7aiLOdDpZo4p5d1kr3VMNjVNM+L9ECpTntPhyrglOX018Q8xcNRh2/z+G6NsiiV4t0YxFdk/ICTQ== Jesse.Williamson@suse.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXsBAd8JfyMT352SjfhZ2Nj3gR2PkeyPS9yB6fKck5CW0Ns1EcUAOIFYw7a1R6zCDJbkkXZDRgM1W6v/UBgpOm69HaNCZuqx19mQRNgo1lhw6hDHc+9754wOmIbgu3gInIS/XPvGzAkGwBW3rWlOaNurY4Y/PH1QyiaujD1Ii9X2Q6yRP8hsDrccual36gngY6wV3G4xKbS8kTY4qQekHkgFT+ZDHv9ibliDfAxtzm5X8N25sICpvcnQkG+Co3QP2XADEhoQrs8L7QekKYPlX8zq6rJBO0cGYW50E58LJk7s67n7QQPCy04xeAf9KOZZBSMHTI8eSFHjb8B034IkHH jwilliamson@linux-z0c6", "key_options": null, "keyfile": "/home/jwilliamson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jwilliamson", "validate_certs": true} 2019-11-29T11:06:22.856 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-11-29T11:06:23.011 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-11-29T11:06:23.013 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-11-29T11:06:23.202 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-11-29T11:06:23.349 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-11-29T11:06:23.559 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-11-29T11:06:23.561 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-11-29T11:06:23.686 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-11-29T11:06:23.912 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-11-29T11:06:24.017 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-11-29T11:06:24.019 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-11-29T11:06:24.269 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-11-29T11:06:24.333 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-11-29T11:06:24.591 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC3aWYoBDxRh13e+qver2xcmCtNsiVEt9NkN1cntdUxVbjfWgb+E/b54kPW456tswoVaYS+3xI5DYTPTN1tMhx8jnrgVKI2HhIYC3SO0+V+HNofzU1p7v5CfZxwyupXOgdfHmEWKOepPop8qWon8wZVUkj2t80w8Ifxh5gLVigGquc4kvMLyy6nBI0k6UGeRr9lSKmWLvzudWP70cWij3ynwxEeIuurU0HI9tDmjaVW5dJuu6IZHDhJrCpeUZBQIDqjH0pg1i1uXXwwiLI9NW8jWuALREDeMxAG4IgOZtoXdS7J9vVXHxKpa8ofb+qYNXI7B3OTLgjSS2ziiU1KEIr wanjun.lp@alibaba-inc.com", "key_options": null, "keyfile": "/home/liupan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liupan", "validate_certs": true} 2019-11-29T11:06:24.706 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-11-29T11:06:24.922 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-11-29T11:06:25.068 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC3aWYoBDxRh13e+qver2xcmCtNsiVEt9NkN1cntdUxVbjfWgb+E/b54kPW456tswoVaYS+3xI5DYTPTN1tMhx8jnrgVKI2HhIYC3SO0+V+HNofzU1p7v5CfZxwyupXOgdfHmEWKOepPop8qWon8wZVUkj2t80w8Ifxh5gLVigGquc4kvMLyy6nBI0k6UGeRr9lSKmWLvzudWP70cWij3ynwxEeIuurU0HI9tDmjaVW5dJuu6IZHDhJrCpeUZBQIDqjH0pg1i1uXXwwiLI9NW8jWuALREDeMxAG4IgOZtoXdS7J9vVXHxKpa8ofb+qYNXI7B3OTLgjSS2ziiU1KEIr wanjun.lp@alibaba-inc.com", "key_options": null, "keyfile": "/home/liupan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liupan", "validate_certs": true} 2019-11-29T11:06:25.258 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0PusmXbjqduPTosGmqFsi2cUjRIiWmMnATY/vuawopBxRNykGnyeh1MfVLgkulM6zBJ3cDZxLsZYykpsHGvaXgLL5r6bEOOKPu8sSMwaTfZWNewue6xa1vOYAggeWOt8KTd0IO/pZZfvFcJ80tjtcNvY8zOJ9TO5W7RpglfJhPneC9Kr726yZ4RX3Hqlx+nIMzrSlvZVtwsKikanImvGLDulRiHqgE5BcqGvDF1yCSIq4OYF2i2Sk+dnPnQtN2uePsxPYKikd0B/rua5q+hDDAz2I36NhaUIhLteE6acWJK5Z2s0o1rWAS5XMbF4qYM3nbaE1Au/nXHcjO3sf6mj3 amarangone@macair.local", "key_options": null, "keyfile": "/home/amarangone/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amarangone", "validate_certs": true} 2019-11-29T11:06:25.396 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-11-29T11:06:25.564 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-29T11:06:25.753 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0PusmXbjqduPTosGmqFsi2cUjRIiWmMnATY/vuawopBxRNykGnyeh1MfVLgkulM6zBJ3cDZxLsZYykpsHGvaXgLL5r6bEOOKPu8sSMwaTfZWNewue6xa1vOYAggeWOt8KTd0IO/pZZfvFcJ80tjtcNvY8zOJ9TO5W7RpglfJhPneC9Kr726yZ4RX3Hqlx+nIMzrSlvZVtwsKikanImvGLDulRiHqgE5BcqGvDF1yCSIq4OYF2i2Sk+dnPnQtN2uePsxPYKikd0B/rua5q+hDDAz2I36NhaUIhLteE6acWJK5Z2s0o1rWAS5XMbF4qYM3nbaE1Au/nXHcjO3sf6mj3 amarangone@macair.local", "key_options": null, "keyfile": "/home/amarangone/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amarangone", "validate_certs": true} 2019-11-29T11:06:25.873 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIGGibBaZykvZPLLQm/UIDUlqVxeyJvZEOO+/7pjqX3O47nilNILLnv/oWhQN1S5eGVev9lnTlTxqI0Mt0N/B9vbCxoxx3ujsSgc+FZL58rMBN4uaTvHhYgqOYo3ht6fciRqSvcqeOGNKU8HnXS0/C8bJrYoKUTnmaaZjQ0bSw/AMsAIxu3bZmWT+HfgVaMh9PjPgC+XnDypEcRkOLbeEIs+0s1TgSR7Hs/vEuVtLH3Ph4WZwKVKQhfugmbblLLNvHvTy0q5afdh2RQ9MEaUgxXMgVq4hJ/JcFNZp+ZU5irOPQosZfjSLWJKx/LH+Ye4/8nINi08w0MZgcK79CwukJ kmroz@suse.com", "key_options": null, "keyfile": "/home/kmroz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kmroz", "validate_certs": true} 2019-11-29T11:06:26.084 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-29T11:06:26.212 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-11-29T11:06:26.406 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIGGibBaZykvZPLLQm/UIDUlqVxeyJvZEOO+/7pjqX3O47nilNILLnv/oWhQN1S5eGVev9lnTlTxqI0Mt0N/B9vbCxoxx3ujsSgc+FZL58rMBN4uaTvHhYgqOYo3ht6fciRqSvcqeOGNKU8HnXS0/C8bJrYoKUTnmaaZjQ0bSw/AMsAIxu3bZmWT+HfgVaMh9PjPgC+XnDypEcRkOLbeEIs+0s1TgSR7Hs/vEuVtLH3Ph4WZwKVKQhfugmbblLLNvHvTy0q5afdh2RQ9MEaUgxXMgVq4hJ/JcFNZp+ZU5irOPQosZfjSLWJKx/LH+Ye4/8nINi08w0MZgcK79CwukJ kmroz@suse.com", "key_options": null, "keyfile": "/home/kmroz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kmroz", "validate_certs": true} 2019-11-29T11:06:26.508 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-11-29T11:06:26.737 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-11-29T11:06:26.835 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-11-29T11:06:27.082 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-11-29T11:06:27.151 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-11-29T11:06:27.396 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-11-29T11:06:27.513 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-11-29T11:06:27.754 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-11-29T11:06:27.844 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-11-29T11:06:28.106 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-11-29T11:06:28.108 INFO:teuthology.task.ansible.out: 2019-11-29T11:06:28.194 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1IrqEe8zPM6DWf8Y+WLRU+TwX8Xzki1CQDRwK/w/F7qxAuh9cbX8tCTkZUt0L3pUjGpM1C3bkTzLHaN4bxK1RQ/klB7C3AXjNOJSPJSjcn/N3YuKU3sKWuQkXnQTwqUdTzHuZ8lQWywFItmwQR2Y6T+xG/Ecig+vdU5lwQ479gtt/lB9buI+T10Lefk2bUiYq1PmKSCIF4KtsIdmSazmIQyvsbsBVblMEezCC0auwo7P7SdfSaobf4x4ZZyJfYkVxiL6eGJ8vmBSbJFrcVMkAO07wBtpn/gQa3Dt4xUdXXt7zBizpMmRPLm8m3Pl2kIJhqrIa1cFXPRr5GJ0YXzR mu3e@Mu3e\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNisR+EK2sG/E9dSeoEYCfgzIunFnuu+SFWGyWR3K6Q5Evt7bVAsGnjglNXdVgLbGV4Il+Z+FFgtR/oJgtUZ7G0IwWKG+nlx06hOu2DObsgM7gcIlhrf7SNsI13Bit0v5dQkyLwcmpdwioQyUx5hnx8UamqVo7lLtKsHS8melz7JE/+dnGwIMJLB1Ot/8PIRAb+/NRXpnD/lje5u8idr43bhreuJ6Lell+h8X2vomuvLlA5XjAoROY6L5qKoM3+bD4QXbU006VHg/7wl9tpZIi9XwAuHfaktsmfks+u3eYFCd9nYUKN20zbEecoAA1rOrCF1qhgwTYpa/Rmh7Iy53p mu3e@thinkpad-t510", "key_options": null, "keyfile": "/home/adamyanova/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adamyanova", "validate_certs": true} 2019-11-29T11:06:28.443 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-11-29T11:06:28.528 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-11-29T11:06:28.805 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1IrqEe8zPM6DWf8Y+WLRU+TwX8Xzki1CQDRwK/w/F7qxAuh9cbX8tCTkZUt0L3pUjGpM1C3bkTzLHaN4bxK1RQ/klB7C3AXjNOJSPJSjcn/N3YuKU3sKWuQkXnQTwqUdTzHuZ8lQWywFItmwQR2Y6T+xG/Ecig+vdU5lwQ479gtt/lB9buI+T10Lefk2bUiYq1PmKSCIF4KtsIdmSazmIQyvsbsBVblMEezCC0auwo7P7SdfSaobf4x4ZZyJfYkVxiL6eGJ8vmBSbJFrcVMkAO07wBtpn/gQa3Dt4xUdXXt7zBizpMmRPLm8m3Pl2kIJhqrIa1cFXPRr5GJ0YXzR mu3e@Mu3e\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNisR+EK2sG/E9dSeoEYCfgzIunFnuu+SFWGyWR3K6Q5Evt7bVAsGnjglNXdVgLbGV4Il+Z+FFgtR/oJgtUZ7G0IwWKG+nlx06hOu2DObsgM7gcIlhrf7SNsI13Bit0v5dQkyLwcmpdwioQyUx5hnx8UamqVo7lLtKsHS8melz7JE/+dnGwIMJLB1Ot/8PIRAb+/NRXpnD/lje5u8idr43bhreuJ6Lell+h8X2vomuvLlA5XjAoROY6L5qKoM3+bD4QXbU006VHg/7wl9tpZIi9XwAuHfaktsmfks+u3eYFCd9nYUKN20zbEecoAA1rOrCF1qhgwTYpa/Rmh7Iy53p mu3e@thinkpad-t510", "key_options": null, "keyfile": "/home/adamyanova/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adamyanova", "validate_certs": true} 2019-11-29T11:06:28.858 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-11-29T11:06:29.131 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-11-29T11:06:29.185 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-11-29T11:06:29.456 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-11-29T11:06:29.549 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-11-29T11:06:29.790 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-11-29T11:06:29.865 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-11-29T11:06:30.128 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-11-29T11:06:30.153 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-11-29T11:06:30.438 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-11-29T11:06:30.537 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-11-29T11:06:30.759 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-11-29T11:06:30.864 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-11-29T11:06:31.109 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-11-29T11:06:31.233 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-11-29T11:06:31.382 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-11-29T11:06:31.589 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-11-29T11:06:31.715 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-11-29T11:06:31.944 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-11-29T11:06:32.029 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-11-29T11:06:32.269 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-11-29T11:06:32.384 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-11-29T11:06:32.600 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-11-29T11:06:32.743 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-11-29T11:06:32.949 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-11-29T11:06:33.089 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-11-29T11:06:33.321 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-11-29T11:06:33.461 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-11-29T11:06:33.663 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-11-29T11:06:33.664 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-11-29T11:06:33.790 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-11-29T11:06:33.994 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-11-29T11:06:33.996 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-11-29T11:06:34.146 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-11-29T11:06:34.148 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-11-29T11:06:34.311 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEJa/U9zUfZ7YFl0kwU7Mx3rHPn2O4sdh1J3XBUgoVdrJ4Qnh7i+QhN+fw0eaBWQE2GIsGEX6YWAL5xJ7EuYZyas8OVLNdP9cJIr79M51d8dG/ywqBpgcFnx7qhCSua3sKmP6GCTWGD9Y0eWL2Z6y9t3amb8dPXviMNqKGDz/DaDtp5bPaoG8mDHQ9QeADYYk3tx6V7+phIBXrI2GKt0BBlJeuZysiUI/XWO46JdMyHvCzvzcdquD446aoWS44waDGj501sgnHVQ7gwf4P1+vFOSEHzkWhbenpJmmTA26f4dim+Jo5JpBW/Ma5mMBvORFhH/Kt+S9P9sPCiF1shWc3 d@d", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-11-29T11:06:34.490 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-11-29T11:06:34.491 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-11-29T11:06:34.689 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-11-29T11:06:34.828 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEJa/U9zUfZ7YFl0kwU7Mx3rHPn2O4sdh1J3XBUgoVdrJ4Qnh7i+QhN+fw0eaBWQE2GIsGEX6YWAL5xJ7EuYZyas8OVLNdP9cJIr79M51d8dG/ywqBpgcFnx7qhCSua3sKmP6GCTWGD9Y0eWL2Z6y9t3amb8dPXviMNqKGDz/DaDtp5bPaoG8mDHQ9QeADYYk3tx6V7+phIBXrI2GKt0BBlJeuZysiUI/XWO46JdMyHvCzvzcdquD446aoWS44waDGj501sgnHVQ7gwf4P1+vFOSEHzkWhbenpJmmTA26f4dim+Jo5JpBW/Ma5mMBvORFhH/Kt+S9P9sPCiF1shWc3 d@d", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-11-29T11:06:35.052 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-11-29T11:06:35.115 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-11-29T11:06:35.383 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-11-29T11:06:35.460 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-11-29T11:06:35.711 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-11-29T11:06:35.797 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-11-29T11:06:36.066 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-11-29T11:06:36.130 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-11-29T11:06:36.404 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-11-29T11:06:36.479 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-11-29T11:06:36.767 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-11-29T11:06:36.810 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-11-29T11:06:37.116 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-11-29T11:06:37.171 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-11-29T11:06:37.452 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-11-29T11:06:37.515 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-11-29T11:06:37.795 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-11-29T11:06:37.848 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-11-29T11:06:38.151 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-11-29T11:06:38.177 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-11-29T11:06:38.471 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-11-29T11:06:38.493 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-11-29T11:06:38.819 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-11-29T11:06:38.851 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-11-29T11:06:39.167 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-11-29T11:06:39.180 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-11-29T11:06:39.182 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-29T11:06:39.494 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-11-29T11:06:39.496 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-29T11:06:39.505 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-11-29T11:06:39.850 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-11-29T11:06:39.925 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-11-29T11:06:39.956 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:39 +0000 (0:00:38.439) 0:01:32.187 ******* 2019-11-29T11:06:39.997 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-29T11:06:39.999 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.008 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.009 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.020 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.031 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.032 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.043 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.054 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.065 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.181 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-11-29T11:06:40.282 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:40 +0000 (0:00:00.325) 0:01:32.513 ******* 2019-11-29T11:06:40.324 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.333 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.343 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.354 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.355 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.366 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.377 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.378 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.388 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.399 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-11-29T11:06:40.411 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:40.759 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224c 2019-11-29T11:06:40.772 INFO:teuthology.task.ansible.out:a01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0 2019-11-29T11:06:40.777 INFO:teuthology.task.ansible.out:fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.104 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9 2019-11-29T11:06:41.106 INFO:teuthology.task.ansible.out:', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "key_options": null, "keyfile": "/home/sbillah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sbillah", "validate_certs": true} 2019-11-29T11:06:41.421 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "key_options": null, "keyfile": "/home/sbillah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sbillah", "validate_certs": true} 2019-11-29T11:06:41.435 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-11-29T11:06:41.712 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-11-29T11:06:41.761 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-11-29T11:06:41.772 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.773 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.784 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.795 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.806 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.817 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.828 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.849 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.865 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.867 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.876 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.877 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.889 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.902 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.912 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.922 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.924 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.934 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.950 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.970 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.983 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.984 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.995 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:41.996 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.007 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.019 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.030 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.041 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.043 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.053 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-11-29T11:06:42.054 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.065 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.068 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.077 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.100 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.104 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.112 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.113 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.115 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.125 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.126 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.139 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.151 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.207 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.c 2019-11-29T11:06:42.214 INFO:teuthology.task.ansible.out:om] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (ite 2019-11-29T11:06:42.215 INFO:teuthology.task.ansible.out:m={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.228 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.234 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.246 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.259 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.264 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.275 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.276 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.279 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.291 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.292 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.294 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.296 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.297 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.299 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.315 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.322 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.323 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.327 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.342 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.344 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.355 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.357 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.358 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.360 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.373 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.377 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.389 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.390 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.392 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.394 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.405 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.418 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.420 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.421 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.422 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.423 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.436 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.438 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.450 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.452 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.463 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.464 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.468 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.470 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.471 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.482 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.494 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.495 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.496 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.507 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.507 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.509 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.520 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.524 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.529 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.533 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.545 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.546 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.563 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.564 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.576 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.578 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.579 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.591 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.592 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.594 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.596 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.598 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.600 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.611 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.612 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.624 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.626 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.640 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.641 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.642 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.654 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.655 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.667 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.669 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.670 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.682 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.699 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.701 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.702 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.704 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.706 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.722 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.744 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.746 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.768 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.792 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.793 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.809 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.872 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.873 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.937 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:42 +0000 (0:00:02.655) 0:01:35.169 ******* 2019-11-29T11:06:42.944 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.945 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.947 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.948 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.966 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.974 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:42.976 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.016 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:43 +0000 (0:00:00.078) 0:01:35.247 ******* 2019-11-29T11:06:43.023 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.028 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.030 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.031 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.033 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.052 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.057 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.110 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.114 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.115 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.126 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.138 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.150 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.151 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.162 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.175 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.181 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.260 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:43 +0000 (0:00:00.244) 0:01:35.491 ******* 2019-11-29T11:06:43.300 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-11-29T11:06:43.302 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:43.338 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:43 +0000 (0:00:00.077) 0:01:35.569 ******* 2019-11-29T11:06:43.365 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-11-29T11:06:43.366 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-11-29T11:06:43.680 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-11-29T11:06:43.699 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-11-29T11:06:43.988 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-11-29T11:06:44.041 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-11-29T11:06:44.314 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-11-29T11:06:44.364 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-11-29T11:06:44.660 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-11-29T11:06:44.693 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-11-29T11:06:44.971 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-11-29T11:06:45.025 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-11-29T11:06:45.297 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-11-29T11:06:45.360 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-11-29T11:06:45.648 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-11-29T11:06:45.706 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-11-29T11:06:45.979 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-11-29T11:06:46.002 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-11-29T11:06:46.312 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-11-29T11:06:46.322 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-11-29T11:06:46.626 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-11-29T11:06:46.635 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-11-29T11:06:46.972 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-11-29T11:06:46.974 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-11-29T11:06:47.295 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-11-29T11:06:47.324 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-11-29T11:06:47.584 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-11-29T11:06:47.675 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-11-29T11:06:47.906 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-11-29T11:06:48.012 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-11-29T11:06:48.255 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-11-29T11:06:48.366 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-11-29T11:06:48.586 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-11-29T11:06:48.692 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-11-29T11:06:48.911 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-11-29T11:06:49.011 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-11-29T11:06:49.221 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-11-29T11:06:49.364 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-11-29T11:06:49.544 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-11-29T11:06:49.701 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-11-29T11:06:49.858 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-11-29T11:06:50.026 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-11-29T11:06:50.153 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-11-29T11:06:50.376 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-11-29T11:06:50.419 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-11-29T11:06:50.733 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-11-29T11:06:50.752 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-11-29T11:06:51.059 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-11-29T11:06:51.062 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-11-29T11:06:51.371 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-11-29T11:06:51.393 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-11-29T11:06:51.697 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-11-29T11:06:51.698 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-11-29T11:06:52.022 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-11-29T11:06:52.035 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-11-29T11:06:52.088 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:52 +0000 (0:00:08.749) 0:01:44.319 ******* 2019-11-29T11:06:52.362 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-11-29T11:06:52.714 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-11-29T11:06:52.810 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:52 +0000 (0:00:00.722) 0:01:45.041 ******* 2019-11-29T11:06:53.221 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-11-29T11:06:53.224 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "checksum": "50650ef8f5fa65d26a4b4e7fa94b1571b873d967", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "f08a845f35203e506ea1deb151ba8443", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025612.12-45678685066627/source", "state": "file", "uid": 0} 2019-11-29T11:06:53.269 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:53 +0000 (0:00:00.457) 0:01:45.499 ******* 2019-11-29T11:06:53.295 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-11-29T11:06:53.296 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => { "msg": "Host smithi163.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-11-29T11:06:53.337 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:53 +0000 (0:00:00.068) 0:01:45.567 ******* 2019-11-29T11:06:53.385 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-11-29T11:06:53.387 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-11-29T11:06:53.448 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:53 +0000 (0:00:00.105) 0:01:45.673 ******* 2019-11-29T11:06:53.493 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:53.556 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:53 +0000 (0:00:00.089) 0:01:45.762 ******* 2019-11-29T11:06:53.969 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-11-29T11:06:53.970 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "checksum": "50650ef8f5fa65d26a4b4e7fa94b1571b873d967", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "f08a845f35203e506ea1deb151ba8443", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025612.84-191362591389976/source", "state": "file", "uid": 0} 2019-11-29T11:06:54.008 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.477) 0:01:46.240 ******* 2019-11-29T11:06:54.016 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.098128", "end": "2019-11-29 11:06:53.959315", "rc": 0, "start": "2019-11-29 11:06:53.861187", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-11-29T11:06:54.062 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.052) 0:01:46.292 ******* 2019-11-29T11:06:54.066 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-11-29T11:06:54.068 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => { "msg": "Host smithi069.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-11-29T11:06:54.112 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.051) 0:01:46.343 ******* 2019-11-29T11:06:54.119 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-11-29T11:06:54.121 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.173 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.054) 0:01:46.398 ******* TASK [common : Including major version specific variables.] ******************** 2019-11-29T11:06:54.181 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-11-29T11:06:54.215 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.048) 0:01:46.446 ******* 2019-11-29T11:06:54.220 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-11-29T11:06:54.222 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.266 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.051) 0:01:46.497 ******* 2019-11-29T11:06:54.272 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-11-29T11:06:54.272 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.312 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.045) 0:01:46.543 ******* 2019-11-29T11:06:54.326 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.360 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.047) 0:01:46.591 ******* 2019-11-29T11:06:54.400 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-11-29T11:06:54.402 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.456 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.094) 0:01:46.685 ******* 2019-11-29T11:06:54.496 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-11-29T11:06:54.497 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.539 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.084) 0:01:46.770 ******* 2019-11-29T11:06:54.568 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.612 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.072) 0:01:46.843 ******* 2019-11-29T11:06:54.661 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-11-29T11:06:54.664 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:54.707 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.095) 0:01:46.939 ******* 2019-11-29T11:06:54.752 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.805 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.097) 0:01:47.036 ******* 2019-11-29T11:06:54.811 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-11-29T11:06:54.812 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.112416", "end": "2019-11-29 11:06:54.750110", "rc": 0, "start": "2019-11-29 11:06:54.637694", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-11-29T11:06:54.851 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.046) 0:01:47.082 ******* 2019-11-29T11:06:54.858 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-11-29T11:06:54.860 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.901 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.049) 0:01:47.132 ******* 2019-11-29T11:06:54.919 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:54.947 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.046) 0:01:47.179 ******* 2019-11-29T11:06:54.954 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-11-29T11:06:54.957 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:54.995 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:54 +0000 (0:00:00.047) 0:01:47.226 ******* 2019-11-29T11:06:55.002 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-11-29T11:06:55.002 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.043 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.047) 0:01:47.274 ******* 2019-11-29T11:06:55.058 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.089 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.046) 0:01:47.320 ******* 2019-11-29T11:06:55.098 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-11-29T11:06:55.100 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.139 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.050) 0:01:47.371 ******* 2019-11-29T11:06:55.146 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-11-29T11:06:55.147 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:55.184 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.044) 0:01:47.415 ******* 2019-11-29T11:06:55.192 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-11-29T11:06:55.194 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.233 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.048) 0:01:47.464 ******* 2019-11-29T11:06:55.239 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-11-29T11:06:55.240 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.280 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.047) 0:01:47.511 ******* 2019-11-29T11:06:55.295 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.330 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.047) 0:01:47.559 ******* 2019-11-29T11:06:55.334 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-11-29T11:06:55.336 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.382 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.053) 0:01:47.613 ******* 2019-11-29T11:06:55.390 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-11-29T11:06:55.392 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.435 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.053) 0:01:47.667 ******* 2019-11-29T11:06:55.444 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-11-29T11:06:55.447 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.499 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.063) 0:01:47.730 ******* 2019-11-29T11:06:55.505 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-11-29T11:06:55.506 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:55.508 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:55.546 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.046) 0:01:47.777 ******* 2019-11-29T11:06:55.554 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-11-29T11:06:55.555 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.598 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.051) 0:01:47.829 ******* 2019-11-29T11:06:55.603 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-11-29T11:06:55.605 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.643 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.045) 0:01:47.874 ******* 2019-11-29T11:06:55.649 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-11-29T11:06:55.651 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:55.702 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.056) 0:01:47.930 ******* 2019-11-29T11:06:55.708 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.748 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.046) 0:01:47.976 ******* 2019-11-29T11:06:55.755 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.801 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.056) 0:01:48.033 ******* 2019-11-29T11:06:55.811 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-11-29T11:06:55.814 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:55.860 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.058) 0:01:48.092 ******* 2019-11-29T11:06:55.868 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-11-29T11:06:55.870 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.912 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.051) 0:01:48.143 ******* 2019-11-29T11:06:55.917 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:55.957 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:55 +0000 (0:00:00.044) 0:01:48.188 ******* 2019-11-29T11:06:55.963 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-11-29T11:06:55.964 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.001 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.044) 0:01:48.233 ******* 2019-11-29T11:06:56.007 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-11-29T11:06:56.009 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:56.061 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.059) 0:01:48.293 ******* 2019-11-29T11:06:56.072 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-11-29T11:06:56.074 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.113 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.051) 0:01:48.345 ******* 2019-11-29T11:06:56.119 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-11-29T11:06:56.121 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.160 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.046) 0:01:48.391 ******* 2019-11-29T11:06:56.168 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-11-29T11:06:56.169 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.211 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.051) 0:01:48.442 ******* 2019-11-29T11:06:56.218 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-11-29T11:06:56.221 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.266 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.055) 0:01:48.498 ******* 2019-11-29T11:06:56.280 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.319 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.052) 0:01:48.551 ******* 2019-11-29T11:06:56.325 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-11-29T11:06:56.327 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.373 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.053) 0:01:48.604 ******* 2019-11-29T11:06:56.380 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-11-29T11:06:56.381 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:56.419 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.045) 0:01:48.650 ******* 2019-11-29T11:06:56.429 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-11-29T11:06:56.431 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.487 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.067) 0:01:48.718 ******* 2019-11-29T11:06:56.497 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-11-29T11:06:56.556 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.068) 0:01:48.787 ******* 2019-11-29T11:06:56.566 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:56.608 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.052) 0:01:48.840 ******* 2019-11-29T11:06:56.619 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.660 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.049) 0:01:48.889 ******* 2019-11-29T11:06:56.669 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.713 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.053) 0:01:48.943 ******* 2019-11-29T11:06:56.752 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-11-29T11:06:56.754 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.796 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.080) 0:01:49.023 ******* 2019-11-29T11:06:56.820 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-11-29T11:06:56.822 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.862 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.070) 0:01:49.093 ******* 2019-11-29T11:06:56.889 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-11-29T11:06:56.890 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:56.940 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:56 +0000 (0:00:00.078) 0:01:49.172 ******* 2019-11-29T11:06:56.969 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-11-29T11:06:56.971 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:57.029 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.088) 0:01:49.260 ******* 2019-11-29T11:06:57.039 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-11-29T11:06:57.040 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:06:57.084 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.055) 0:01:49.315 ******* 2019-11-29T11:06:57.090 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-11-29T11:06:57.091 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:57.146 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.061) 0:01:49.377 ******* 2019-11-29T11:06:57.187 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-11-29T11:06:57.190 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-29T11:06:57.241 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.082) 0:01:49.459 ******* 2019-11-29T11:06:57.258 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-11-29T11:06:57.305 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.071) 0:01:49.531 ******* 2019-11-29T11:06:57.353 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:06:57.400 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.100) 0:01:49.631 ******* 2019-11-29T11:06:57.755 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:06:57.799 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:57 +0000 (0:00:00.394) 0:01:50.025 ******* 2019-11-29T11:06:58.451 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-11-29T11:06:58.828 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-11-29T11:06:59.418 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-11-29T11:06:59.462 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:59 +0000 (0:00:01.667) 0:01:51.693 ******* 2019-11-29T11:06:59.804 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-11-29T11:06:59.847 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:06:59 +0000 (0:00:00.385) 0:01:52.079 ******* 2019-11-29T11:07:00.606 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi163.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025619.5-66375252898262/source", "state": "file", "uid": 0} 2019-11-29T11:07:01.061 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025619.89-149007751287290/source", "state": "file", "uid": 0} 2019-11-29T11:07:01.789 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025620.62-134098485097488/source", "state": "file", "uid": 0} 2019-11-29T11:07:01.836 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:01 +0000 (0:00:01.988) 0:01:54.067 ******* 2019-11-29T11:07:02.295 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575025621.08-182720176756650/source", "state": "file", "uid": 0} 2019-11-29T11:07:02.342 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:02 +0000 (0:00:00.506) 0:01:54.573 ******* 2019-11-29T11:07:02.511 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-11-29T11:07:02.512 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-11-29T11:07:02.683 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-11-29T11:07:02.840 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-29T11:07:02.842 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-11-29T11:07:02.885 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:02 +0000 (0:00:00.542) 0:01:55.116 ******* 2019-11-29T11:07:02.979 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-29T11:07:03.032 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:03 +0000 (0:00:00.146) 0:01:55.263 ******* 2019-11-29T11:07:15.058 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-11-29T11:07:15.059 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:11.719862", "end": "2019-11-29 11:07:15.026016", "rc": 0, "start": "2019-11-29 11:07:03.306154", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.facebook.net\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.facebook.net", " * extras: mirrors.greenmountainaccess.net", " * updates: mirror.facebook.net", "Metadata Cache Created"]} 2019-11-29T11:07:15.106 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:15 +0000 (0:00:12.073) 0:02:07.337 ******* 2019-11-29T11:07:15.475 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-11-29T11:07:15.642 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.443506", "end": "2019-11-29 11:07:15.617546", "rc": 0, "start": "2019-11-29 11:07:03.174040", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.facebook.net\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.facebook.net", " * extras: mirrors.greenmountainaccess.net", " * updates: mirror.facebook.net", "Metadata Cache Created"]} 2019-11-29T11:07:15.686 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:15 +0000 (0:00:00.580) 0:02:07.917 ******* 2019-11-29T11:07:15.787 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-11-29T11:07:15.789 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-29T11:07:15.831 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:15 +0000 (0:00:00.144) 0:02:08.062 ******* 2019-11-29T11:07:15.858 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-29T11:07:15.860 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:15.898 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:15 +0000 (0:00:00.066) 0:02:08.129 ******* 2019-11-29T11:07:15.943 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:15.974 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:15 +0000 (0:00:00.066) 0:02:08.196 ******* 2019-11-29T11:07:15.994 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.034 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.067) 0:02:08.263 ******* 2019-11-29T11:07:16.039 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-11-29T11:07:16.042 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-11-29T11:07:16.062 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-29T11:07:16.064 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.103 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.071) 0:02:08.334 ******* 2019-11-29T11:07:16.130 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-11-29T11:07:16.132 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.174 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.070) 0:02:08.405 ******* 2019-11-29T11:07:16.216 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-11-29T11:07:16.217 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.268 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.083) 0:02:08.489 ******* 2019-11-29T11:07:16.302 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.343 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.083) 0:02:08.573 ******* 2019-11-29T11:07:16.362 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-11-29T11:07:16.362 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-29T11:07:16.364 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-11-29T11:07:16.366 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.401 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.059) 0:02:08.632 ******* 2019-11-29T11:07:16.437 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.034) 0:02:08.667 ******* 2019-11-29T11:07:16.463 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-29T11:07:16.464 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.502 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.064) 0:02:08.732 ******* 2019-11-29T11:07:16.528 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-11-29T11:07:16.533 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.567 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.065) 0:02:08.798 ******* 2019-11-29T11:07:16.601 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.634 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.066) 0:02:08.864 ******* 2019-11-29T11:07:16.677 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.709 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.076) 0:02:08.941 ******* 2019-11-29T11:07:16.748 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.784 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.074) 0:02:09.015 ******* 2019-11-29T11:07:16.822 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-11-29T11:07:16.824 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.859 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.074) 0:02:09.090 ******* 2019-11-29T11:07:16.896 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-11-29T11:07:16.898 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:16.967 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:16 +0000 (0:00:00.107) 0:02:09.197 ******* 2019-11-29T11:07:16.993 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-11-29T11:07:16.995 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:17.033 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:17 +0000 (0:00:00.067) 0:02:09.265 ******* 2019-11-29T11:07:18.529 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-11-29T11:07:18.531 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.791905", "end": "2019-11-29 11:07:18.500456", "rc": 0, "start": "2019-11-29 11:07:16.708551", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.191161", "end": "2019-11-29 11:07:18.500448", "rc": 0, "start": "2019-11-29 11:07:17.309287", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:07:18.690 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:18 +0000 (0:00:01.656) 0:02:10.921 ******* 2019-11-29T11:07:18.724 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:18 +0000 (0:00:00.033) 0:02:10.955 ******* 2019-11-29T11:07:19.480 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-11-29T11:07:19.480 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-11-29T11:07:19.524 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:19 +0000 (0:00:00.795) 0:02:11.750 ******* 2019-11-29T11:07:19.559 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:19.584 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:19 +0000 (0:00:00.065) 0:02:11.816 ******* 2019-11-29T11:07:19.610 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-11-29T11:07:19.612 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:19.649 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:19 +0000 (0:00:00.064) 0:02:11.880 ******* 2019-11-29T11:07:19.676 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:19.711 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:19 +0000 (0:00:00.062) 0:02:11.943 ******* 2019-11-29T11:07:19.738 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:19.779 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:19 +0000 (0:00:00.067) 0:02:12.010 ******* 2019-11-29T11:07:20.474 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-11-29T11:07:20.475 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-11-29T11:07:20.514 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.734) 0:02:12.745 ******* 2019-11-29T11:07:20.541 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-11-29T11:07:20.543 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:20.588 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.074) 0:02:12.819 ******* 2019-11-29T11:07:20.614 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-11-29T11:07:20.616 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:20.665 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.076) 0:02:12.896 ******* 2019-11-29T11:07:20.693 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-11-29T11:07:20.694 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add krb5 config file] ******************************************* ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-11-29T11:07:20.733 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.068) 0:02:12.965 ******* 2019-11-29T11:07:20.768 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.034) 0:02:13.000 ******* 2019-11-29T11:07:20.794 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-11-29T11:07:20.796 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:20.833 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:20 +0000 (0:00:00.064) 0:02:13.064 ******* 2019-11-29T11:07:21.684 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-11-29T11:07:21.727 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:21 +0000 (0:00:00.886) 0:02:13.951 ******* 2019-11-29T11:07:24.742 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-11-29T11:07:24.793 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:24 +0000 (0:00:03.066) 0:02:17.017 ******* 2019-11-29T11:07:25.547 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-11-29T11:07:25.577 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:25 +0000 (0:00:00.790) 0:02:17.808 ******* 2019-11-29T11:07:25.732 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi069.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-11-29T11:07:26.534 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-11-29T11:07:26.642 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-11-29T11:07:27.460 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-11-29T11:07:27.473 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-11-29T11:07:27.520 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:27 +0000 (0:00:01.942) 0:02:19.751 ******* 2019-11-29T11:07:27.856 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-11-29T11:07:27.857 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:07:27.897 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:27 +0000 (0:00:00.377) 0:02:20.128 ******* 2019-11-29T11:07:28.413 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi163.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-11-29T11:07:28.486 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:28 +0000 (0:00:00.589) 0:02:20.717 ******* 2019-11-29T11:07:28.800 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:07:28.840 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:28 +0000 (0:00:00.354) 0:02:21.071 ******* 2019-11-29T11:07:28.846 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-11-29T11:07:28.847 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-11-29T11:07:29.746 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} ok: [smithi069.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-11-29T11:07:30.608 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-11-29T11:07:30.677 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-11-29T11:07:30.716 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:30 +0000 (0:00:01.876) 0:02:22.947 ******* 2019-11-29T11:07:30.756 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-11-29T11:07:30.758 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:30.794 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:30 +0000 (0:00:00.077) 0:02:23.025 ******* 2019-11-29T11:07:30.836 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-11-29T11:07:30.838 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-11-29T11:07:30.881 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:30 +0000 (0:00:00.087) 0:02:23.112 ******* 2019-11-29T11:07:31.224 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi069.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-11-29T11:07:31.276 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:31 +0000 (0:00:00.389) 0:02:23.502 ******* 2019-11-29T11:07:31.326 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-11-29T11:07:31.328 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:31.341 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:31.363 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:31.377 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:31.418 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:31 +0000 (0:00:00.147) 0:02:23.650 ******* 2019-11-29T11:07:31.467 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-11-29T11:07:31.470 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-11-29T11:07:31.522 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:31 +0000 (0:00:00.103) 0:02:23.753 ******* 2019-11-29T11:07:31.550 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:31.587 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:31 +0000 (0:00:00.065) 0:02:23.819 ******* 2019-11-29T11:07:31.627 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-11-29T11:07:31.666 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:31 +0000 (0:00:00.078) 0:02:23.897 ******* 2019-11-29T11:07:31.976 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-11-29T11:07:31.978 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-11-29T11:07:32.021 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:32 +0000 (0:00:00.355) 0:02:24.252 ******* 2019-11-29T11:07:32.071 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-11-29T11:07:32.073 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:32.083 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:32.099 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:32.106 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-29T11:07:32.156 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:32 +0000 (0:00:00.131) 0:02:24.384 ******* 2019-11-29T11:07:34.749 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-11-29T11:07:34.804 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:34 +0000 (0:00:02.640) 0:02:27.024 ******* 2019-11-29T11:07:34.831 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-11-29T11:07:34.832 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:34.871 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:34 +0000 (0:00:00.076) 0:02:27.101 ******* 2019-11-29T11:07:35.446 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-11-29T11:07:35.496 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:35 +0000 (0:00:00.625) 0:02:27.727 ******* 2019-11-29T11:07:35.536 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-11-29T11:07:35.538 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:35.594 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:35 +0000 (0:00:00.097) 0:02:27.825 ******* 2019-11-29T11:07:35.795 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-11-29T11:07:35.848 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:35 +0000 (0:00:00.245) 0:02:28.070 ******* 2019-11-29T11:07:36.547 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-11-29T11:07:36.614 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:36 +0000 (0:00:00.775) 0:02:28.846 ******* 2019-11-29T11:07:37.581 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-11-29T11:07:37.611 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:37 +0000 (0:00:00.996) 0:02:29.842 ******* 2019-11-29T11:07:37.649 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:37.688 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:37 +0000 (0:00:00.076) 0:02:29.919 ******* 2019-11-29T11:07:37.804 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-11-29T11:07:37.855 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:37 +0000 (0:00:00.166) 0:02:30.086 ******* 2019-11-29T11:07:37.895 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:37.951 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:37 +0000 (0:00:00.096) 0:02:30.182 ******* 2019-11-29T11:07:38.030 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:07:38.072 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.120) 0:02:30.302 ******* 2019-11-29T11:07:38.293 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:07:38.330 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.258) 0:02:30.561 ******* 2019-11-29T11:07:38.435 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-11-29T11:07:38.437 INFO:teuthology.task.ansible.out:fatal: [smithi163.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-11-29T11:07:38.496 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.166) 0:02:30.728 ******* 2019-11-29T11:07:38.535 INFO:teuthology.task.ansible.out: 2019-11-29T11:07:38.537 INFO:teuthology.task.ansible.out:TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:38.597 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.099) 0:02:30.827 ******* 2019-11-29T11:07:38.712 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi069.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-11-29T11:07:38.750 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.154) 0:02:30.981 ******* 2019-11-29T11:07:38.777 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-11-29T11:07:38.778 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:07:38.814 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:38 +0000 (0:00:00.064) 0:02:31.046 ******* 2019-11-29T11:07:39.503 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-11-29T11:07:39.503 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-11-29T11:07:39.545 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:39 +0000 (0:00:00.730) 0:02:31.776 ******* 2019-11-29T11:07:39.747 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-11-29T11:07:39.789 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:39 +0000 (0:00:00.244) 0:02:32.020 ******* 2019-11-29T11:07:40.482 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-11-29T11:07:40.485 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2019-11-29 11:04:54 UTC", "ActiveEnterTimestampMonotonic": "21830978", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target var-run.mount time-sync.target basic.target network.target systemd-journald.socket system.slice nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Fri 2019-11-29 11:04:54 UTC", "AssertTimestampMonotonic": "21830274", "Before": "multi-user.target xdm.service getty@tty1.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2019-11-29 11:04:54 UTC", "ConditionTimestampMonotonic": "21830272", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8966", "ExecMainStartTimestamp": "Fri 2019-11-29 11:04:54 UTC", "ExecMainStartTimestampMonotonic": "21830930", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-11-29T11:07:40.490 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Fri 2019-11-29 11:04:54 UTC] ; stop_time=[n/a] ; pid=8966 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2019-11-29 11:04:54 UTC", "InactiveExitTimestampMonotonic": "21830978", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8966", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-11-29T11:07:40.492 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Fri 2019-11-29 11:04:54 UTC", "WatchdogTimestampMonotonic": "21830960", "WatchdogUSec": "0"}} 2019-11-29T11:07:40.541 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:40 +0000 (0:00:00.748) 0:02:32.769 ******* 2019-11-29T11:07:40.552 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2019-11-29 11:04:41 UTC", "ActiveEnterTimestampMonotonic": "19481386", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice time-sync.target nss-lookup.target basic.target network.target local-fs.target systemd-journald.socket var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Fri 2019-11-29 11:04:41 UTC", "AssertTimestampMonotonic": "19480679", "Before": "xdm.service multi-user.target shutdown.target getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2019-11-29 11:04:41 UTC", "ConditionTimestampMonotonic": "19480678", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8968", "ExecMainStartTimestamp": "Fri 2019-11-29 11:04:41 UTC", "ExecMainStartTimestampMonotonic": "19481332", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-11-29T11:07:40.560 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Fri 2019-11-29 11:04:41 UTC] ; stop_time=[n/a] ; pid=8968 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2019-11-29 11:04:41 UTC", "InactiveExitTimestampMonotonic": "19481386", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8968", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-11-29T11:07:40.570 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Fri 2019-11-29 11:04:41 UTC", "WatchdogTimestampMonotonic": "19481369", "WatchdogUSec": "0"}} 2019-11-29T11:07:40.601 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:40 +0000 (0:00:00.060) 0:02:32.829 ******* 2019-11-29T11:07:40.858 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004679", "end": "2019-11-29 11:07:40.834057", "rc": 0, "start": "2019-11-29 11:07:40.829378", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-11-29T11:07:40.909 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:40 +0000 (0:00:00.310) 0:02:33.140 ******* 2019-11-29T11:07:40.949 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004086", "end": "2019-11-29 11:07:40.908526", "rc": 0, "start": "2019-11-29 11:07:40.904440", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-11-29T11:07:41.004 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:40 +0000 (0:00:00.090) 0:02:33.230 ******* 2019-11-29T11:07:41.436 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-11-29T11:07:41.504 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:41 +0000 (0:00:00.504) 0:02:33.735 ******* 2019-11-29T11:07:41.515 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-11-29T11:07:41.564 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:41 +0000 (0:00:00.058) 0:02:33.794 ******* 2019-11-29T11:07:43.946 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2019-11-29T11:07:43.985 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:43 +0000 (0:00:02.422) 0:02:36.217 ******* 2019-11-29T11:07:44.051 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2019-11-29T11:07:44.100 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:44 +0000 (0:00:00.114) 0:02:36.331 ******* 2019-11-29T11:07:45.359 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-11-29T11:07:45.360 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-11-29T11:07:45.402 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:45 +0000 (0:00:01.301) 0:02:37.633 ******* 2019-11-29T11:07:45.429 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-11-29T11:07:45.472 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:45 +0000 (0:00:00.069) 0:02:37.702 ******* 2019-11-29T11:07:53.591 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-11-29T11:07:53.593 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.853769", "end": "2019-11-29 11:07:53.541473", "failed_when_result": false, "rc": 0, "start": "2019-11-29 11:07:45.687704", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-11-29T11:07:53.631 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:53 +0000 (0:00:08.159) 0:02:45.861 ******* 2019-11-29T11:07:53.732 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.942052", "end": "2019-11-29 11:07:53.692668", "failed_when_result": false, "rc": 0, "start": "2019-11-29 11:07:45.750616", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-11-29T11:07:53.783 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:53 +0000 (0:00:00.150) 0:02:46.012 ******* 2019-11-29T11:07:54.506 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-11-29T11:07:54.563 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:54 +0000 (0:00:00.779) 0:02:46.792 ******* 2019-11-29T11:07:54.768 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-11-29T11:07:54.846 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:54 +0000 (0:00:00.270) 0:02:47.063 ******* 2019-11-29T11:07:54.876 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.011557", "end": "2019-11-29 11:07:54.844184", "rc": 0, "start": "2019-11-29 11:07:54.832627", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-11-29T11:07:54.909 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:54 +0000 (0:00:00.077) 0:02:47.140 ******* 2019-11-29T11:07:55.156 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.020399", "end": "2019-11-29 11:07:55.124786", "rc": 0, "start": "2019-11-29 11:07:55.104387", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-11-29T11:07:55.206 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:55 +0000 (0:00:00.297) 0:02:47.437 ******* 2019-11-29T11:07:55.235 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-11-29T11:07:55.237 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012064", "end": "2019-11-29 11:07:55.203531", "rc": 0, "start": "2019-11-29 11:07:55.191467", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:07:55.292 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:55 +0000 (0:00:00.073) 0:02:47.511 ******* 2019-11-29T11:07:55.532 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.014616", "end": "2019-11-29 11:07:55.497390", "rc": 0, "start": "2019-11-29 11:07:55.482774", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:07:55.570 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:07:55 +0000 (0:00:00.290) 0:02:47.802 ******* 2019-11-29T11:08:03.461 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-11-29T11:08:03.463 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.848147", "end": "2019-11-29 11:08:03.435527", "rc": 0, "start": "2019-11-29 11:07:55.587380", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:08:03.499 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:03 +0000 (0:00:07.929) 0:02:55.731 ******* 2019-11-29T11:08:03.864 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-11-29T11:08:03.909 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:03 +0000 (0:00:00.398) 0:02:56.129 ******* 2019-11-29T11:08:03.935 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-11-29T11:08:03.937 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-11-29T11:08:03.976 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:03 +0000 (0:00:00.078) 0:02:56.208 ******* 2019-11-29T11:08:04.003 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-11-29T11:08:04.004 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.112268", "end": "2019-11-29 11:08:03.966111", "rc": 0, "start": "2019-11-29 11:07:55.853843", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:08:04.044 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.068) 0:02:56.276 ******* 2019-11-29T11:08:04.050 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-11-29T11:08:04.050 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-29T11:08:04.091 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.047) 0:02:56.323 ******* 2019-11-29T11:08:04.140 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-11-29T11:08:04.142 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-11-29T11:08:04.184 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.092) 0:02:56.416 ******* 2019-11-29T11:08:04.235 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-11-29T11:08:04.237 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-29T11:08:04.279 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.094) 0:02:56.510 ******* 2019-11-29T11:08:04.405 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-11-29T11:08:04.407 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-11-29T11:08:04.449 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.170) 0:02:56.681 ******* 2019-11-29T11:08:04.494 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-11-29T11:08:04.495 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-11-29T11:08:04.543 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.093) 0:02:56.774 ******* 2019-11-29T11:08:04.579 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-11-29T11:08:04.581 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-29T11:08:04.621 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.078) 0:02:56.853 ******* 2019-11-29T11:08:04.629 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-11-29T11:08:04.630 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-29T11:08:04.675 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.053) 0:02:56.906 ******* 2019-11-29T11:08:04.681 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-11-29T11:08:04.683 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-11-29T11:08:04.738 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.063) 0:02:56.969 ******* 2019-11-29T11:08:04.780 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-11-29T11:08:04.782 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-29T11:08:04.832 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:04 +0000 (0:00:00.091) 0:02:57.061 ******* 2019-11-29T11:08:04.994 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-11-29T11:08:04.996 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-11-29T11:08:05.037 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.204) 0:02:57.265 ******* 2019-11-29T11:08:05.062 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:05.101 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.067) 0:02:57.332 ******* 2019-11-29T11:08:05.127 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-11-29T11:08:05.129 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-29T11:08:05.169 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.068) 0:02:57.401 ******* 2019-11-29T11:08:05.207 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.035) 0:02:57.436 ******* 2019-11-29T11:08:05.536 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi069.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-11-29T11:08:05.572 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.366) 0:02:57.803 ******* 2019-11-29T11:08:05.607 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.034) 0:02:57.838 ******* 2019-11-29T11:08:05.636 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:05.680 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.070) 0:02:57.908 ******* 2019-11-29T11:08:05.716 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-11-29T11:08:05.718 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:05.754 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:05 +0000 (0:00:00.077) 0:02:57.985 ******* 2019-11-29T11:08:06.092 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-11-29T11:08:06.093 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-11-29T11:08:06.128 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:06 +0000 (0:00:00.374) 0:02:58.360 ******* 2019-11-29T11:08:06.634 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-11-29T11:08:06.635 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-11-29T11:08:06.684 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:06 +0000 (0:00:00.555) 0:02:58.915 ******* 2019-11-29T11:08:06.721 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-11-29T11:08:06.723 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi069"}, "changed": false} 2019-11-29T11:08:06.768 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:06 +0000 (0:00:00.084) 0:02:59.000 ******* 2019-11-29T11:08:07.149 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-11-29T11:08:07.203 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:07 +0000 (0:00:00.434) 0:02:59.434 ******* 2019-11-29T11:08:07.251 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-11-29T11:08:07.252 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi163"}, "changed": false} 2019-11-29T11:08:07.300 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:07 +0000 (0:00:00.094) 0:02:59.529 ******* 2019-11-29T11:08:07.786 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-11-29T11:08:07.788 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi069.front.sepia.ceph.com", "ansible_hostname": "smithi069", "ansible_nodename": "smithi069"}, "changed": true, "name": "smithi069"} 2019-11-29T11:08:07.830 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:07 +0000 (0:00:00.531) 0:03:00.061 ******* 2019-11-29T11:08:07.837 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi163.front.sepia.ceph.com", "ansible_hostname": "smithi163", "ansible_nodename": "smithi163"}, "changed": true, "name": "smithi163"} 2019-11-29T11:08:07.888 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:07 +0000 (0:00:00.058) 0:03:00.120 ******* 2019-11-29T11:08:08.682 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-11-29T11:08:08.683 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-11-29T11:08:08.740 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:08 +0000 (0:00:00.851) 0:03:00.971 ******* 2019-11-29T11:08:08.757 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-11-29T11:08:08.799 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:08 +0000 (0:00:00.058) 0:03:01.030 ******* 2019-11-29T11:08:09.667 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-11-29T11:08:09.669 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-11-29T11:08:09.710 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:09 +0000 (0:00:00.906) 0:03:01.936 ******* 2019-11-29T11:08:09.733 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-11-29T11:08:09.774 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:09 +0000 (0:00:00.067) 0:03:02.004 ******* 2019-11-29T11:08:10.572 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi069.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-11-29T11:08:10.610 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:10 +0000 (0:00:00.837) 0:03:02.841 ******* 2019-11-29T11:08:10.648 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-11-29T11:08:10.719 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:10 +0000 (0:00:00.104) 0:03:02.946 ******* 2019-11-29T11:08:11.590 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-11-29T11:08:11.591 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-11-29T11:08:11.637 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:11 +0000 (0:00:00.918) 0:03:03.864 ******* 2019-11-29T11:08:11.670 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-11-29T11:08:11.719 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:11 +0000 (0:00:00.085) 0:03:03.949 ******* 2019-11-29T11:08:11.978 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-29T11:08:12.020 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.301) 0:03:04.251 ******* 2019-11-29T11:08:12.067 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-29T11:08:12.099 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.078) 0:03:04.330 ******* 2019-11-29T11:08:12.367 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-11-29T11:08:12.369 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-11-29T11:08:12.408 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.309) 0:03:04.639 ******* 2019-11-29T11:08:12.414 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-11-29T11:08:12.455 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.047) 0:03:04.686 ******* 2019-11-29T11:08:12.771 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-11-29T11:08:12.772 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-11-29T11:08:12.827 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.366) 0:03:05.053 ******* 2019-11-29T11:08:12.831 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-11-29T11:08:12.882 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:12 +0000 (0:00:00.059) 0:03:05.112 ******* 2019-11-29T11:08:13.791 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-29T11:08:13.792 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-11-29T11:08:13.837 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:13 +0000 (0:00:00.956) 0:03:06.068 ******* 2019-11-29T11:08:13.848 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-11-29T11:08:13.912 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:13 +0000 (0:00:00.074) 0:03:06.143 ******* 2019-11-29T11:08:14.796 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-11-29T11:08:14.877 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:14 +0000 (0:00:00.960) 0:03:07.104 ******* 2019-11-29T11:08:14.923 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-11-29T11:08:14.985 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:14 +0000 (0:00:00.112) 0:03:07.216 ******* 2019-11-29T11:08:17.341 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-11-29T11:08:17.342 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2019-11-29T11:08:17.344 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.123951", "end": "2019-11-29 11:08:17.306674", "rc": 0, "start": "2019-11-29 11:08:15.182723", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:08:17.396 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:17 +0000 (0:00:02.399) 0:03:09.615 ******* changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.073820", "end": "2019-11-29 11:08:17.346830", "rc": 0, "start": "2019-11-29 11:08:15.273010", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:08:17.430 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:17 +0000 (0:00:00.046) 0:03:09.662 ******* 2019-11-29T11:08:17.436 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-11-29T11:08:17.479 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:17 +0000 (0:00:00.046) 0:03:09.708 ******* 2019-11-29T11:08:17.520 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:17 +0000 (0:00:00.043) 0:03:09.752 ******* 2019-11-29T11:08:18.460 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi069.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-11-29T11:08:18.482 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-11-29T11:08:19.383 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-11-29T11:08:19.424 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:01.903) 0:03:11.655 ******* 2019-11-29T11:08:19.440 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-11-29T11:08:19.492 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.061) 0:03:11.716 ******* 2019-11-29T11:08:19.494 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.538 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.052) 0:03:11.769 ******* 2019-11-29T11:08:19.545 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.591 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.053) 0:03:11.822 ******* 2019-11-29T11:08:19.597 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-11-29T11:08:19.598 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.599 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.643 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.051) 0:03:11.874 ******* 2019-11-29T11:08:19.648 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.650 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.690 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.046) 0:03:11.921 ******* 2019-11-29T11:08:19.695 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-11-29T11:08:19.701 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.734 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.043) 0:03:11.965 ******* 2019-11-29T11:08:19.741 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:19.780 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.046) 0:03:12.011 ******* 2019-11-29T11:08:19.786 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:08:19.787 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-11-29T11:08:19.828 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.047) 0:03:12.059 ******* 2019-11-29T11:08:19.833 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-11-29T11:08:19.876 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:19 +0000 (0:00:00.046) 0:03:12.105 ******* 2019-11-29T11:08:20.329 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-11-29T11:08:20.331 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-11-29T11:08:20.379 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:20 +0000 (0:00:00.501) 0:03:12.606 ******* 2019-11-29T11:08:20.382 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-11-29T11:08:20.423 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:20 +0000 (0:00:00.047) 0:03:12.654 ******* 2019-11-29T11:08:21.044 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.322968", "end": "2019-11-29 11:08:20.997798", "rc": 0, "start": "2019-11-29 11:08:20.674830", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.facebook.net", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.greenmountainaccess.net", " * updates: mirror.facebook.net", "No unfinished transactions left."]} 2019-11-29T11:08:21.088 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.665) 0:03:13.320 ******* 2019-11-29T11:08:21.109 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.348367", "end": "2019-11-29 11:08:21.073524", "rc": 0, "start": "2019-11-29 11:08:20.725157", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.facebook.net", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.greenmountainaccess.net", " * updates: mirror.facebook.net", "No unfinished transactions left."]} 2019-11-29T11:08:21.159 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.071) 0:03:13.391 ******* 2019-11-29T11:08:21.482 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-11-29T11:08:21.485 INFO:teuthology.task.ansible.out:fatal: [smithi163.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.056483", "end": "2019-11-29 11:08:21.445223", "msg": "non-zero return code", "rc": 1, "start": "2019-11-29 11:08:21.388740", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-11-29T11:08:21.522 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.362) 0:03:13.754 ******* 2019-11-29T11:08:21.543 INFO:teuthology.task.ansible.out:fatal: [smithi069.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.036079", "end": "2019-11-29 11:08:21.504854", "msg": "non-zero return code", "rc": 1, "start": "2019-11-29 11:08:21.468775", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-11-29T11:08:21.582 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.059) 0:03:13.813 ******* 2019-11-29T11:08:21.589 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-11-29T11:08:21.591 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:21.633 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.051) 0:03:13.865 ******* 2019-11-29T11:08:21.640 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:21.681 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:21 +0000 (0:00:00.047) 0:03:13.913 ******* 2019-11-29T11:08:31.986 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-11-29T11:08:31.987 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-11-29T11:08:31.988 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-11-29T11:08:31.990 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-29T11:08:32.030 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:32 +0000 (0:00:10.348) 0:03:24.261 ******* 2019-11-29T11:08:32.414 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-11-29T11:08:32.416 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-11-29T11:08:32.418 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-29T11:08:32.479 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:32 +0000 (0:00:00.438) 0:03:24.700 ******* 2019-11-29T11:08:35.395 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-11-29T11:08:35.436 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:35 +0000 (0:00:02.967) 0:03:27.667 ******* 2019-11-29T11:08:35.896 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-11-29T11:08:35.939 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:35 +0000 (0:00:00.502) 0:03:28.170 ******* 2019-11-29T11:08:53.518 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-11-29T11:08:53.520 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-11-29T11:08:53.522 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-11-29T11:08:53.524 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependenc 2019-11-29T11:08:53.541 INFO:teuthology.task.ansible.out:ies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 20 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-11-29T11:08:53.543 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-29T11:08:53.563 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:53 +0000 (0:00:17.622) 0:03:45.792 ******* 2019-11-29T11:08:54.176 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-11-29T11:08:54.184 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-11-29T11:08:54.186 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.greenmountainaccess.net\n * updates: mirror.facebook.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependenc 2019-11-29T11:08:54.187 INFO:teuthology.task.ansible.out:ies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 21 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-11-29T11:08:54.189 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x8 2019-11-29T11:08:54.190 INFO:teuthology.task.ansible.out:6_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-29T11:08:54.240 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:54 +0000 (0:00:00.677) 0:03:46.470 ******* 2019-11-29T11:08:54.313 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-11-29T11:08:54.314 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-11-29T11:08:54.315 INFO:teuthology.task.ansible.out: 2019-11-29T11:08:54.359 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:54 +0000 (0:00:00.119) 0:03:46.590 ******* 2019-11-29T11:08:54.916 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-11-29T11:08:54.952 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:54 +0000 (0:00:00.586) 0:03:47.176 ******* 2019-11-29T11:08:54.983 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-29T11:08:54.984 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:55.035 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.090) 0:03:47.266 ******* 2019-11-29T11:08:55.056 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-11-29T11:08:55.081 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.045) 0:03:47.312 ******* 2019-11-29T11:08:55.471 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service polkit.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service iptables.service ipset.service shutdown.target ebtables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": " 2019-11-29T11:08:55.473 INFO:teuthology.task.ansible.out:yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPr 2019-11-29T11:08:55.474 INFO:teuthology.task.ansible.out:iority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-29T11:08:55.507 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.426) 0:03:47.738 ******* 2019-11-29T11:08:55.653 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-11-29T11:08:55.691 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.183) 0:03:47.922 ******* 2019-11-29T11:08:55.729 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-29T11:08:55.730 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:55.769 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.077) 0:03:48.000 ******* 2019-11-29T11:08:55.908 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi163.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-11-29T11:08:55.947 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:55 +0000 (0:00:00.176) 0:03:48.177 ******* 2019-11-29T11:08:56.238 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice dbus.service polkit.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target ebtables.service ipset.service ip6tables.service iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": " 2019-11-29T11:08:56.239 INFO:teuthology.task.ansible.out:yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPr 2019-11-29T11:08:56.241 INFO:teuthology.task.ansible.out:iority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-29T11:08:56.279 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:56 +0000 (0:00:00.328) 0:03:48.506 ******* 2019-11-29T11:08:56.679 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-11-29T11:08:56.681 INFO:teuthology.task.ansible.out:fatal: [smithi069.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-11-29T11:08:56.720 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:56 +0000 (0:00:00.446) 0:03:48.952 ******* 2019-11-29T11:08:56.782 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-11-29T11:08:56.821 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:56 +0000 (0:00:00.100) 0:03:49.052 ******* 2019-11-29T11:08:56.850 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:56.888 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:56 +0000 (0:00:00.066) 0:03:49.119 ******* 2019-11-29T11:08:56.915 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-11-29T11:08:56.963 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:56 +0000 (0:00:00.075) 0:03:49.195 ******* 2019-11-29T11:08:56.992 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-11-29T11:08:56.993 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.035 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.068) 0:03:49.263 ******* 2019-11-29T11:08:57.060 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-11-29T11:08:57.061 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.112 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.078) 0:03:49.342 ******* 2019-11-29T11:08:57.128 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-11-29T11:08:57.129 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-11-29T11:08:57.175 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.064) 0:03:49.406 ******* 2019-11-29T11:08:57.182 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-11-29T11:08:57.182 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.184 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.225 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.049) 0:03:49.456 ******* 2019-11-29T11:08:57.230 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-11-29T11:08:57.232 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.272 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.046) 0:03:49.503 ******* 2019-11-29T11:08:57.278 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-11-29T11:08:57.279 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.318 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.045) 0:03:49.548 ******* 2019-11-29T11:08:57.322 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-11-29T11:08:57.361 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.043) 0:03:49.592 ******* 2019-11-29T11:08:57.375 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-11-29T11:08:57.412 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.051) 0:03:49.643 ******* 2019-11-29T11:08:57.418 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-11-29T11:08:57.419 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.460 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.047) 0:03:49.691 ******* 2019-11-29T11:08:57.469 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.512 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.052) 0:03:49.743 ******* 2019-11-29T11:08:57.518 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-11-29T11:08:57.519 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.568 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.055) 0:03:49.799 ******* 2019-11-29T11:08:57.575 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.635 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.063) 0:03:49.863 ******* 2019-11-29T11:08:57.639 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-11-29T11:08:57.641 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.687 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.052) 0:03:49.915 ******* 2019-11-29T11:08:57.690 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-11-29T11:08:57.691 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.741 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.046) 0:03:49.962 ******* TASK [testnode : Install python-apt] ******************************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.778 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.047) 0:03:50.009 ******* 2019-11-29T11:08:57.784 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-29T11:08:57.786 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.825 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.047) 0:03:50.056 ******* 2019-11-29T11:08:57.832 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-11-29T11:08:57.873 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.047) 0:03:50.104 ******* 2019-11-29T11:08:57.880 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-11-29T11:08:57.882 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.925 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.048) 0:03:50.153 ******* 2019-11-29T11:08:57.927 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-11-29T11:08:57.929 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:57.971 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:57 +0000 (0:00:00.044) 0:03:50.198 ******* 2019-11-29T11:08:57.976 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.018 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.050) 0:03:50.249 ******* 2019-11-29T11:08:58.023 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-11-29T11:08:58.024 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.066 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.048) 0:03:50.297 ******* 2019-11-29T11:08:58.073 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-11-29T11:08:58.075 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.113 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.047) 0:03:50.345 ******* 2019-11-29T11:08:58.120 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-11-29T11:08:58.127 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.158 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.045) 0:03:50.390 ******* 2019-11-29T11:08:58.165 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-11-29T11:08:58.167 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.206 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.047) 0:03:50.437 ******* 2019-11-29T11:08:58.215 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-29T11:08:58.217 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.254 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.048) 0:03:50.485 ******* 2019-11-29T11:08:58.264 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.301 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.047) 0:03:50.532 ******* 2019-11-29T11:08:58.307 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-11-29T11:08:58.309 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.347 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.045) 0:03:50.578 ******* 2019-11-29T11:08:58.353 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-11-29T11:08:58.354 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.392 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.044) 0:03:50.623 ******* 2019-11-29T11:08:58.413 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.452 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.055) 0:03:50.678 ******* 2019-11-29T11:08:58.457 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.492 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.044) 0:03:50.723 ******* 2019-11-29T11:08:58.508 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.535 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.043) 0:03:50.766 ******* 2019-11-29T11:08:58.543 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-11-29T11:08:58.545 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.581 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.044) 0:03:50.811 ******* 2019-11-29T11:08:58.597 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.626 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.045) 0:03:50.857 ******* 2019-11-29T11:08:58.636 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.674 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.048) 0:03:50.905 ******* 2019-11-29T11:08:58.679 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-11-29T11:08:58.680 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.717 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.043) 0:03:50.949 ******* 2019-11-29T11:08:58.724 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-11-29T11:08:58.725 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.762 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.044) 0:03:50.993 ******* 2019-11-29T11:08:58.775 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-11-29T11:08:58.777 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.821 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.056) 0:03:51.050 ******* 2019-11-29T11:08:58.825 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.864 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.045) 0:03:51.095 ******* 2019-11-29T11:08:58.869 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-11-29T11:08:58.870 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.910 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.046) 0:03:51.141 ******* 2019-11-29T11:08:58.916 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-11-29T11:08:58.917 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.959 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.045) 0:03:51.186 ******* 2019-11-29T11:08:58.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:58.999 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:58 +0000 (0:00:00.043) 0:03:51.230 ******* 2019-11-29T11:08:59.007 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-11-29T11:08:59.008 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.044 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:51.275 ******* 2019-11-29T11:08:59.051 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-11-29T11:08:59.052 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.091 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.047) 0:03:51.322 ******* 2019-11-29T11:08:59.097 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-29T11:08:59.099 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.136 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:51.367 ******* 2019-11-29T11:08:59.142 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-11-29T11:08:59.143 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.187 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.047) 0:03:51.415 ******* 2019-11-29T11:08:59.190 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-11-29T11:08:59.191 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.229 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.045) 0:03:51.460 ******* 2019-11-29T11:08:59.237 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-11-29T11:08:59.239 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.276 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.045) 0:03:51.505 ******* 2019-11-29T11:08:59.284 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.318 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:51.550 ******* 2019-11-29T11:08:59.324 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-11-29T11:08:59.325 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.364 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.045) 0:03:51.596 ******* 2019-11-29T11:08:59.371 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-11-29T11:08:59.372 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.415 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.049) 0:03:51.645 ******* 2019-11-29T11:08:59.419 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-11-29T11:08:59.420 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.457 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.043) 0:03:51.689 ******* 2019-11-29T11:08:59.465 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-11-29T11:08:59.467 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.508 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.049) 0:03:51.739 ******* 2019-11-29T11:08:59.513 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-29T11:08:59.513 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.552 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:51.783 ******* 2019-11-29T11:08:59.558 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-11-29T11:08:59.559 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.598 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.045) 0:03:51.829 ******* 2019-11-29T11:08:59.603 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-11-29T11:08:59.605 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.644 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.046) 0:03:51.875 ******* 2019-11-29T11:08:59.652 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-11-29T11:08:59.654 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.705 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.060) 0:03:51.936 ******* 2019-11-29T11:08:59.710 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-11-29T11:08:59.713 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.753 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:51.981 ******* 2019-11-29T11:08:59.764 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.795 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:52.025 ******* 2019-11-29T11:08:59.806 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.844 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:52.069 ******* 2019-11-29T11:08:59.845 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.883 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.043) 0:03:52.113 ******* 2019-11-29T11:08:59.890 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.935 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.044) 0:03:52.158 ******* 2019-11-29T11:08:59.953 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-11-29T11:08:59.955 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:08:59.998 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:08:59 +0000 (0:00:00.065) 0:03:52.223 ******* 2019-11-29T11:09:00.019 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-11-29T11:09:00.021 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:00.056 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.064) 0:03:52.287 ******* 2019-11-29T11:09:00.093 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-29T11:09:00.095 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:00.156 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.096) 0:03:52.384 ******* 2019-11-29T11:09:00.190 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-11-29T11:09:00.192 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:00.235 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.082) 0:03:52.467 ******* 2019-11-29T11:09:00.242 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-29T11:09:00.243 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-11-29T11:09:00.284 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.049) 0:03:52.516 ******* 2019-11-29T11:09:00.558 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-11-29T11:09:00.595 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.311) 0:03:52.827 ******* 2019-11-29T11:09:00.612 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-29T11:09:00.613 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:09:00.650 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.054) 0:03:52.881 ******* 2019-11-29T11:09:00.940 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:09:00.991 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:00 +0000 (0:00:00.339) 0:03:53.221 ******* 2019-11-29T11:09:00.997 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-29T11:09:00.998 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-11-29T11:09:01.044 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.053) 0:03:53.274 ******* 2019-11-29T11:09:01.069 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-29T11:09:01.071 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.106 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.062) 0:03:53.337 ******* 2019-11-29T11:09:01.132 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-29T11:09:01.134 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.169 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.063) 0:03:53.400 ******* 2019-11-29T11:09:01.195 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-11-29T11:09:01.197 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.231 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.062) 0:03:53.462 ******* 2019-11-29T11:09:01.257 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-29T11:09:01.261 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.296 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.065) 0:03:53.527 ******* 2019-11-29T11:09:01.328 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.364 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.068) 0:03:53.596 ******* 2019-11-29T11:09:01.399 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.034) 0:03:53.630 ******* 2019-11-29T11:09:01.403 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-29T11:09:01.405 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.448 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.049) 0:03:53.679 ******* 2019-11-29T11:09:01.454 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-29T11:09:01.455 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.492 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.044) 0:03:53.723 ******* 2019-11-29T11:09:01.497 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-29T11:09:01.499 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.536 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.044) 0:03:53.768 ******* 2019-11-29T11:09:01.552 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.580 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.043) 0:03:53.811 ******* 2019-11-29T11:09:01.592 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.627 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.047) 0:03:53.858 ******* 2019-11-29T11:09:01.635 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.673 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.046) 0:03:53.905 ******* 2019-11-29T11:09:01.682 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.725 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.045) 0:03:53.950 ******* 2019-11-29T11:09:01.729 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.768 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.048) 0:03:53.999 ******* 2019-11-29T11:09:01.773 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-29T11:09:01.775 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.811 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.043) 0:03:54.042 ******* 2019-11-29T11:09:01.819 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-11-29T11:09:01.820 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.856 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.044) 0:03:54.087 ******* 2019-11-29T11:09:01.868 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.900 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.043) 0:03:54.131 ******* 2019-11-29T11:09:01.905 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-11-29T11:09:01.906 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.943 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.044) 0:03:54.175 ******* 2019-11-29T11:09:01.951 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:01.987 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:01 +0000 (0:00:00.043) 0:03:54.218 ******* 2019-11-29T11:09:01.993 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.032 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.045) 0:03:54.263 ******* 2019-11-29T11:09:02.037 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-29T11:09:02.039 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.076 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.043) 0:03:54.307 ******* 2019-11-29T11:09:02.081 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-29T11:09:02.083 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.123 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.047) 0:03:54.355 ******* 2019-11-29T11:09:02.138 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-29T11:09:02.140 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.198 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.074) 0:03:54.429 ******* 2019-11-29T11:09:02.207 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-11-29T11:09:02.209 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.248 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.050) 0:03:54.480 ******* 2019-11-29T11:09:02.254 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-11-29T11:09:02.256 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.295 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.046) 0:03:54.527 ******* 2019-11-29T11:09:02.302 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-11-29T11:09:02.304 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.347 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.051) 0:03:54.578 ******* 2019-11-29T11:09:02.355 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.396 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.047) 0:03:54.626 ******* 2019-11-29T11:09:02.403 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-11-29T11:09:02.403 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.439 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.044) 0:03:54.670 ******* 2019-11-29T11:09:02.444 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-29T11:09:02.446 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.482 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.043) 0:03:54.713 ******* 2019-11-29T11:09:02.487 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-11-29T11:09:02.489 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.525 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.043) 0:03:54.757 ******* 2019-11-29T11:09:02.530 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-29T11:09:02.532 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.572 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.046) 0:03:54.803 ******* 2019-11-29T11:09:02.578 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-11-29T11:09:02.578 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.620 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.048) 0:03:54.852 ******* 2019-11-29T11:09:02.626 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-11-29T11:09:02.627 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.665 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.044) 0:03:54.896 ******* 2019-11-29T11:09:02.671 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-11-29T11:09:02.672 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.708 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.043) 0:03:54.940 ******* 2019-11-29T11:09:02.714 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-11-29T11:09:02.716 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.755 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.046) 0:03:54.986 ******* 2019-11-29T11:09:02.760 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-11-29T11:09:02.762 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.800 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.044) 0:03:55.031 ******* 2019-11-29T11:09:02.806 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-11-29T11:09:02.806 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.851 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.051) 0:03:55.083 ******* 2019-11-29T11:09:02.860 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-11-29T11:09:02.861 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.900 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.048) 0:03:55.131 ******* 2019-11-29T11:09:02.905 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-11-29T11:09:02.907 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.943 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.042) 0:03:55.174 ******* 2019-11-29T11:09:02.949 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-11-29T11:09:02.950 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:02.994 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:02 +0000 (0:00:00.045) 0:03:55.219 ******* TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.030 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.042) 0:03:55.262 ******* 2019-11-29T11:09:03.038 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:09:03.040 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-11-29T11:09:03.075 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.044) 0:03:55.306 ******* 2019-11-29T11:09:03.080 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-11-29T11:09:03.082 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.118 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.043) 0:03:55.350 ******* 2019-11-29T11:09:03.124 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-11-29T11:09:03.125 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.136 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-11-29T11:09:03.137 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.147 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-11-29T11:09:03.149 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi163.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.189 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.070) 0:03:55.421 ******* 2019-11-29T11:09:03.195 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.196 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.198 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.238 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.048) 0:03:55.469 ******* 2019-11-29T11:09:03.249 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.283 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.044) 0:03:55.514 ******* 2019-11-29T11:09:03.310 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-11-29T11:09:03.311 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.347 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.064) 0:03:55.579 ******* 2019-11-29T11:09:03.385 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-11-29T11:09:03.424 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.074) 0:03:55.653 ******* 2019-11-29T11:09:03.431 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi163.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005549", "end": "2019-11-29 11:09:03.368889", "rc": 0, "start": "2019-11-29 11:09:03.363340", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-11-29T11:09:03.480 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.057) 0:03:55.711 ******* 2019-11-29T11:09:03.486 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-11-29T11:09:03.488 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi069.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-11-29T11:09:03.499 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.511 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.512 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.553 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.072) 0:03:55.784 ******* 2019-11-29T11:09:03.560 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi069.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.613 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.059) 0:03:55.844 ******* 2019-11-29T11:09:03.622 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-11-29T11:09:03.624 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.662 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.049) 0:03:55.893 ******* 2019-11-29T11:09:03.699 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-11-29T11:09:03.701 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.751 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.089) 0:03:55.983 ******* 2019-11-29T11:09:03.753 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:03.770 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:03.771 INFO:teuthology.task.ansible.out:TASK [testnode : Check for teuthology-lock command] **************************** 2019-11-29T11:09:03.773 INFO:teuthology.task.ansible.out: changed: [smithi069.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005337", "end": "2019-11-29 11:09:03.746065", "rc": 0, "start": "2019-11-29 11:09:03.740728", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-11-29T11:09:03.774 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:03.824 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.066) 0:03:56.050 ******* 2019-11-29T11:09:03.826 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.871 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.046) 0:03:56.096 ******* TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:03.916 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:03 +0000 (0:00:00.050) 0:03:56.147 ******* 2019-11-29T11:09:03.974 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-11-29T11:09:03.976 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:04.015 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.099) 0:03:56.246 ******* 2019-11-29T11:09:04.042 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-11-29T11:09:04.044 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:04.089 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.074) 0:03:56.321 ******* 2019-11-29T11:09:04.115 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-11-29T11:09:04.117 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:04.152 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.062) 0:03:56.383 ******* 2019-11-29T11:09:04.376 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-11-29T11:09:04.378 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-11-29T11:09:04.423 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.270) 0:03:56.654 ******* 2019-11-29T11:09:04.472 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi163.front.sepia.ceph.com] => (item={u'block_used': 13034935, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289549, u'size_available': 930977992704, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289549, "block_size": 4096, "block_total": 240324484, "block_used": 13034935, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930977992704, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-11-29T11:09:04.505 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.081) 0:03:56.736 ******* 2019-11-29T11:09:04.507 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:04.573 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-11-29T11:09:04.574 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-11-29T11:09:04.615 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.108) 0:03:56.845 ******* 2019-11-29T11:09:04.654 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-11-29T11:09:04.654 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item={u'block_used': 13034913, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289571, u'size_available': 930978082816, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289571, "block_size": 4096, "block_total": 240324484, "block_used": 13034913, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930978082816, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-11-29T11:09:04.700 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.086) 0:03:56.932 ******* 2019-11-29T11:09:04.843 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-11-29T11:09:04.845 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014263", "end": "2019-11-29 11:09:04.818685", "rc": 0, "start": "2019-11-29 11:09:04.804422", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:04.890 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.189) 0:03:57.122 ******* 2019-11-29T11:09:04.931 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-11-29T11:09:04.932 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'block_used': 13034935, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289549, u'size_available': 930977992704, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289549, "block_size": 4096, "block_total": 240324484, "block_used": 13034935, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930977992704, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:04.972 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:04 +0000 (0:00:00.081) 0:03:57.203 ******* 2019-11-29T11:09:04.999 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-11-29T11:09:05.044 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:05 +0000 (0:00:00.069) 0:03:57.273 ******* 2019-11-29T11:09:05.048 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-11-29T11:09:05.050 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014316", "end": "2019-11-29 11:09:05.002072", "rc": 0, "start": "2019-11-29 11:09:04.987756", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:05.088 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:05 +0000 (0:00:00.047) 0:03:57.320 ******* 2019-11-29T11:09:05.107 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-11-29T11:09:05.108 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFMX', u'wwn-0x5000c50092924fd4'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XFMX', u'wwn': u'0x5000c50092924fd4', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFMX-part1', u'wwn-0x5000c50092924fd4-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFMX", "wwn-0x5000c50092924fd4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFMX-part1", "wwn-0x5000c50092924fd4-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFMX", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-29T11:09:05.110 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50092924fd4"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:05.130 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-11-29T11:09:05.132 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={u'block_used': 13034913, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289571, u'size_available': 930978082816, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289571, "block_size": 4096, "block_total": 240324484, "block_used": 13034913, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930978082816, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:05.171 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:05 +0000 (0:00:00.081) 0:03:57.401 ******* 2019-11-29T11:09:05.197 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-11-29T11:09:05.237 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:05 +0000 (0:00:00.067) 0:03:57.468 ******* 2019-11-29T11:09:05.275 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-11-29T11:09:05.276 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QJH7', u'wwn-0x5000c50091e1a1b4'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QJH7', u'wwn': u'0x5000c50091e1a1b4', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QJH7-part1', u'wwn-0x5000c50091e1a1b4-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJH7", "wwn-0x5000c50091e1a1b4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJH7-part1", "wwn-0x5000c50091e1a1b4-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QJH7", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-29T11:09:05.277 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50091e1a1b4"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:06.916 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-109fR2-aBSb-PprC-G8ok-NA3M-0e0V-e6kamh', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300MD400BGN', u'nvme-nvme.8086-435646543632333330304d4434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.498426", "end": "2019-11-29 11:09:06.873820", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-109fR2-aBSb-PprC-G8ok-NA3M-0e0V-e6kamh", "nvme-INTEL_SSDPEDMD400G4_CVFT623300MD400BGN", "nvme-nvme.8086-435646543632333330304d4434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:05.375394", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2019-11-29T11:09:06.917 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:07.165 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-LMFs3D-BfS9-a2dB-SPFZ-edX9-eCIy-Oc4tOg', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN', u'nvme-nvme.8086-50484654363230343030593034303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.567794", "end": "2019-11-29 11:09:07.125627", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-LMFs3D-BfS9-a2dB-SPFZ-edX9-eCIy-Oc4tOg", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN", "nvme-nvme.8086-50484654363230343030593034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:05.557833", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2019-11-29T11:09:07.167 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:08.306 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltF3hkVFJxSdLabtfEtkUDuL76VUywk7xix'], u'uuids': [u'eb7e31a2-58ca-41af-b7d2-a62a83ef6882']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.079928", "end": "2019-11-29 11:09:08.270719", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltF3hkVFJxSdLabtfEtkUDuL76VUywk7xix"], "labels": [], "masters": [], "uuids": ["eb7e31a2-58ca-41af-b7d2-a62a83ef6882"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:07.190791", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:08.560 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbxwVw3bCHmf7tWhst3mWLP1bttZeed3uK'], u'uuids': [u'71c949e9-1a76-4b7f-be1e-4e0995103a26']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.090289", "end": "2019-11-29 11:09:08.522330", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbxwVw3bCHmf7tWhst3mWLP1bttZeed3uK"], "labels": [], "masters": [], "uuids": ["71c949e9-1a76-4b7f-be1e-4e0995103a26"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:07.432041", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:09.713 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFvj72CEyYFd78m1gAySp7FmDSdhdSBNv4'], u'uuids': [u'e7291326-ff92-441c-b19a-b9bb33c80e3f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.067714", "end": "2019-11-29 11:09:09.668904", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFvj72CEyYFd78m1gAySp7FmDSdhdSBNv4"], "labels": [], "masters": [], "uuids": ["e7291326-ff92-441c-b19a-b9bb33c80e3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:08.601190", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:09.980 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjblS4csXxwYWVoHCpt2arkHMlBaJizbEmr'], u'uuids': [u'332e8bf2-6ccc-40bb-a757-5eb3640220de']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.092893", "end": "2019-11-29 11:09:09.937220", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjblS4csXxwYWVoHCpt2arkHMlBaJizbEmr"], "labels": [], "masters": [], "uuids": ["332e8bf2-6ccc-40bb-a757-5eb3640220de"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:08.844327", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:11.097 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFwKIrsCvCzTeVoP0ZzZv8leSf17PhgkAT'], u'uuids': [u'4774df10-0cbc-4ba4-813f-713d24b70f2e']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.063828", "end": "2019-11-29 11:09:11.055122", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFwKIrsCvCzTeVoP0ZzZv8leSf17PhgkAT"], "labels": [], "masters": [], "uuids": ["4774df10-0cbc-4ba4-813f-713d24b70f2e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:09.991294", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:11.402 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbJHjYM91zz9S2acA2d7vkjzou8uyVqVa7'], u'uuids': [u'201d894d-3403-411a-8155-116e6e595da6']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.095825", "end": "2019-11-29 11:09:11.354454", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbJHjYM91zz9S2acA2d7vkjzou8uyVqVa7"], "labels": [], "masters": [], "uuids": ["201d894d-3403-411a-8155-116e6e595da6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:10.258629", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:12.459 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFLFXTGlsw8mPQymFtggNSdM01RLxdn8ri'], u'uuids': [u'caef99e8-61e7-4d92-aa31-3311e15204ed']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.065147", "end": "2019-11-29 11:09:12.421115", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFLFXTGlsw8mPQymFtggNSdM01RLxdn8ri"], "labels": [], "masters": [], "uuids": ["caef99e8-61e7-4d92-aa31-3311e15204ed"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:11.355968", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:12.799 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbiP11qYj7QcGi4fxzdMzdCGj53WdwT1uY'], u'uuids': [u'bf5b7808-3c48-441a-8cc4-a9e566a32504']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.081666", "end": "2019-11-29 11:09:12.765187", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbiP11qYj7QcGi4fxzdMzdCGj53WdwT1uY"], "labels": [], "masters": [], "uuids": ["bf5b7808-3c48-441a-8cc4-a9e566a32504"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:11.683521", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:13.859 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFyL22YBYSjWfzOG4svuaJQUNeaN0myRRU'], u'uuids': [u'f634f837-6663-43d0-8936-6ce50ce67dfe']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.062941", "end": "2019-11-29 11:09:13.822422", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFyL22YBYSjWfzOG4svuaJQUNeaN0myRRU"], "labels": [], "masters": [], "uuids": ["f634f837-6663-43d0-8936-6ce50ce67dfe"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:12.759481", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:13.932 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:13 +0000 (0:00:08.693) 0:04:06.162 ******* 2019-11-29T11:09:13.983 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFMX', u'wwn-0x5000c50092924fd4'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XFMX', u'wwn': u'0x5000c50092924fd4', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFMX-part1', u'wwn-0x5000c50092924fd4-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFMX", "wwn-0x5000c50092924fd4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFMX-part1", "wwn-0x5000c50092924fd4-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-11-29T11:09:13.984 INFO:teuthology.task.ansible.out:512", "serial": "Z1W5XFMX", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092924fd4"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:14.186 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-11-29T11:09:14.194 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbNcq3BzlaJV1lN1gVQKs5VspmdxEYA82Y'], u'uuids': [u'eeaa3a9b-932e-42b9-bc07-f38e5b6e22bf']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.080688", "end": "2019-11-29 11:09:14.156530", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbNcq3BzlaJV1lN1gVQKs5VspmdxEYA82Y"], "labels": [], "masters": [], "uuids": ["eeaa3a9b-932e-42b9-bc07-f38e5b6e22bf"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:13.075842", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-29T11:09:14.254 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:14 +0000 (0:00:00.323) 0:04:06.485 ******* 2019-11-29T11:09:14.328 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QJH7', u'wwn-0x5000c50091e1a1b4'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QJH7', u'wwn': u'0x5000c50091e1a1b4', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QJH7-part1', u'wwn-0x5000c50091e1a1b4-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJH7", "wwn-0x5000c50091e1a1b4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJH7-part1", "wwn-0x5000c50091e1a1b4-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-11-29T11:09:14.337 INFO:teuthology.task.ansible.out:512", "serial": "Z1W5QJH7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1a1b4"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:14.350 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-109fR2-aBSb-PprC-G8ok-NA3M-0e0V-e6kamh', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300MD400BGN', u'nvme-nvme.8086-435646543632333330304d4434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.061979", "end": "2019-11-29 11:09:14.308643", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-109fR2-aBSb-PprC-G8ok-NA3M-0e0V-e6kamh", "nvme-INTEL_SSDPEDMD400G4_CVFT623300MD400BGN", "nvme-nvme.8086-435646543632333330304d4434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:14.246664", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) co 2019-11-29T11:09:14.352 INFO:teuthology.task.ansible.out:pied, 0.0557337 s, 2.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0557337 s, 2.1 GB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:14.741 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-LMFs3D-BfS9-a2dB-SPFZ-edX9-eCIy-Oc4tOg', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN', u'nvme-nvme.8086-50484654363230343030593034303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087234", "end": "2019-11-29 11:09:14.703668", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-LMFs3D-BfS9-a2dB-SPFZ-edX9-eCIy-Oc4tOg", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN", "nvme-nvme.8086-50484654363230343030593034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:14.616434", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) co 2019-11-29T11:09:14.743 INFO:teuthology.task.ansible.out:pied, 0.0807219 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0807219 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:14.817 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltF3hkVFJxSdLabtfEtkUDuL76VUywk7xix'], u'uuids': [u'eb7e31a2-58ca-41af-b7d2-a62a83ef6882']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.187214", "end": "2019-11-29 11:09:14.781490", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltF3hkVFJxSdLabtfEtkUDuL76VUywk7xix"], "labels": [], "masters": [], "uuids": ["eb7e31a2-58ca-41af-b7d2-a62a83ef6882"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:14.594276", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.181358 s, 636 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.181358 s, 636 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:15.207 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbxwVw3bCHmf7tWhst3mWLP1bttZeed3uK'], u'uuids': [u'71c949e9-1a76-4b7f-be1e-4e0995103a26']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.171115", "end": "2019-11-29 11:09:15.165720", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbxwVw3bCHmf7tWhst3mWLP1bttZeed3uK"], "labels": [], "masters": [], "uuids": ["71c949e9-1a76-4b7f-be1e-4e0995103a26"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:14.994605", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.164465 s, 701 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.164465 s, 701 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:15.344 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFvj72CEyYFd78m1gAySp7FmDSdhdSBNv4'], u'uuids': [u'e7291326-ff92-441c-b19a-b9bb33c80e3f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.202848", "end": "2019-11-29 11:09:15.305093", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFvj72CEyYFd78m1gAySp7FmDSdhdSBNv4"], "labels": [], "masters": [], "uuids": ["e7291326-ff92-441c-b19a-b9bb33c80e3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:15.102245", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.196473 s, 587 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.196473 s, 587 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:15.676 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjblS4csXxwYWVoHCpt2arkHMlBaJizbEmr'], u'uuids': [u'332e8bf2-6ccc-40bb-a757-5eb3640220de']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.168107", "end": "2019-11-29 11:09:15.635552", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjblS4csXxwYWVoHCpt2arkHMlBaJizbEmr"], "labels": [], "masters": [], "uuids": ["332e8bf2-6ccc-40bb-a757-5eb3640220de"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:15.467445", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.162011 s, 712 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.162011 s, 712 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:15.828 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFwKIrsCvCzTeVoP0ZzZv8leSf17PhgkAT'], u'uuids': [u'4774df10-0cbc-4ba4-813f-713d24b70f2e']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.181079", "end": "2019-11-29 11:09:15.794735", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFwKIrsCvCzTeVoP0ZzZv8leSf17PhgkAT"], "labels": [], "masters": [], "uuids": ["4774df10-0cbc-4ba4-813f-713d24b70f2e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:15.613656", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.17492 s, 659 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.17492 s, 659 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:16.152 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbJHjYM91zz9S2acA2d7vkjzou8uyVqVa7'], u'uuids': [u'201d894d-3403-411a-8155-116e6e595da6']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.179830", "end": "2019-11-29 11:09:16.121604", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbJHjYM91zz9S2acA2d7vkjzou8uyVqVa7"], "labels": [], "masters": [], "uuids": ["201d894d-3403-411a-8155-116e6e595da6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:15.941774", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.173206 s, 666 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.173206 s, 666 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:16.349 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFLFXTGlsw8mPQymFtggNSdM01RLxdn8ri'], u'uuids': [u'caef99e8-61e7-4d92-aa31-3311e15204ed']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.194778", "end": "2019-11-29 11:09:16.315214", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFLFXTGlsw8mPQymFtggNSdM01RLxdn8ri"], "labels": [], "masters": [], "uuids": ["caef99e8-61e7-4d92-aa31-3311e15204ed"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:16.120436", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.188377 s, 612 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.188377 s, 612 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:16.656 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbiP11qYj7QcGi4fxzdMzdCGj53WdwT1uY'], u'uuids': [u'bf5b7808-3c48-441a-8cc4-a9e566a32504']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.164761", "end": "2019-11-29 11:09:16.612826", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbiP11qYj7QcGi4fxzdMzdCGj53WdwT1uY"], "labels": [], "masters": [], "uuids": ["bf5b7808-3c48-441a-8cc4-a9e566a32504"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:16.448065", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.158518 s, 728 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.158518 s, 728 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:16.892 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFyL22YBYSjWfzOG4svuaJQUNeaN0myRRU'], u'uuids': [u'f634f837-6663-43d0-8936-6ce50ce67dfe']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.206706", "end": "2019-11-29 11:09:16.853039", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-qeRveTuogNjT9o8C6loRQUrM4axyaltFyL22YBYSjWfzOG4svuaJQUNeaN0myRRU"], "labels": [], "masters": [], "uuids": ["f634f837-6663-43d0-8936-6ce50ce67dfe"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:16.646333", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.199973 s, 577 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.199973 s, 577 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:16.974 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:16 +0000 (0:00:02.719) 0:04:09.205 ******* 2019-11-29T11:09:17.140 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbNcq3BzlaJV1lN1gVQKs5VspmdxEYA82Y'], u'uuids': [u'eeaa3a9b-932e-42b9-bc07-f38e5b6e22bf']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.194031", "end": "2019-11-29 11:09:17.097180", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-ItuQnYwk1tqZ8YOwB1fCavRS2RhbLXjbNcq3BzlaJV1lN1gVQKs5VspmdxEYA82Y"], "labels": [], "masters": [], "uuids": ["eeaa3a9b-932e-42b9-bc07-f38e5b6e22bf"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-29 11:09:16.903149", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.187876 s, 614 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.187876 s, 614 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:17.209 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.231) 0:04:09.436 ******* 2019-11-29T11:09:17.381 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi163.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.091271", "end": "2019-11-29 11:09:17.345486", "rc": 0, "start": "2019-11-29 11:09:17.254215", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:17.433 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.225) 0:04:09.662 ******* 2019-11-29T11:09:17.619 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.079038", "end": "2019-11-29 11:09:17.583216", "rc": 0, "start": "2019-11-29 11:09:17.504178", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:17.671 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.230) 0:04:09.892 ******* 2019-11-29T11:09:17.816 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.056101", "end": "2019-11-29 11:09:17.787808", "rc": 0, "start": "2019-11-29 11:09:17.731707", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:17.860 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.198) 0:04:10.092 ******* 2019-11-29T11:09:17.887 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-11-29T11:09:17.930 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.069) 0:04:10.161 ******* 2019-11-29T11:09:17.957 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-11-29T11:09:17.959 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.002 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:17 +0000 (0:00:00.069) 0:04:10.230 ******* 2019-11-29T11:09:18.025 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-11-29T11:09:18.027 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.076 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.074) 0:04:10.304 ******* 2019-11-29T11:09:18.081 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.069256", "end": "2019-11-29 11:09:18.026755", "rc": 0, "start": "2019-11-29 11:09:17.957499", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:18.139 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.065) 0:04:10.370 ******* 2019-11-29T11:09:18.145 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-29T11:09:18.146 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.191 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.052) 0:04:10.422 ******* 2019-11-29T11:09:18.197 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-11-29T11:09:18.245 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.050) 0:04:10.472 ******* 2019-11-29T11:09:18.247 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-11-29T11:09:18.248 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.287 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.045) 0:04:10.518 ******* 2019-11-29T11:09:18.293 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-11-29T11:09:18.294 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.336 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.048) 0:04:10.566 ******* 2019-11-29T11:09:18.373 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-11-29T11:09:18.374 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.411 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.075) 0:04:10.642 ******* 2019-11-29T11:09:18.437 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-29T11:09:18.439 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.477 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.066) 0:04:10.708 ******* 2019-11-29T11:09:18.515 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-11-29T11:09:18.516 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.554 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.077) 0:04:10.786 ******* 2019-11-29T11:09:18.628 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-11-29T11:09:18.630 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:09:18.667 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.112) 0:04:10.899 ******* 2019-11-29T11:09:18.715 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-11-29T11:09:18.716 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={u'block_used': 13034935, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289549, u'size_available': 930977992704, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289549, "block_size": 4096, "block_total": 240324484, "block_used": 13034935, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930977992704, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.765 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.097) 0:04:10.997 ******* 2019-11-29T11:09:18.792 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-11-29T11:09:18.794 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.831 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.065) 0:04:11.063 ******* 2019-11-29T11:09:18.871 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:09:18.876 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:18.942 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:18 +0000 (0:00:00.109) 0:04:11.173 ******* 2019-11-29T11:09:18.953 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-11-29T11:09:18.954 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-29T11:09:19.015 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.073) 0:04:11.246 ******* 2019-11-29T11:09:19.049 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.071 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.054) 0:04:11.302 ******* 2019-11-29T11:09:19.088 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi069.front.sepia.ceph.com] => (item={u'block_used': 13034913, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227289571, u'size_available': 930978082816, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966955, u'device': u'/dev/sda1', u'inode_used': 88021, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227289571, "block_size": 4096, "block_total": 240324484, "block_used": 13034913, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966955, "inode_total": 61054976, "inode_used": 88021, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930978082816, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.133 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.063) 0:04:11.365 ******* 2019-11-29T11:09:19.141 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-11-29T11:09:19.142 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.184 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.050) 0:04:11.415 ******* 2019-11-29T11:09:19.190 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-11-29T11:09:19.191 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.231 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.046) 0:04:11.462 ******* 2019-11-29T11:09:19.240 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-11-29T11:09:19.242 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.286 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.054) 0:04:11.516 ******* 2019-11-29T11:09:19.291 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:09:19.292 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.351 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.060) 0:04:11.577 ******* 2019-11-29T11:09:19.357 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-11-29T11:09:19.359 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.412 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.066) 0:04:11.643 ******* 2019-11-29T11:09:19.419 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:09:19.420 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.458 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.046) 0:04:11.690 ******* 2019-11-29T11:09:19.513 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-11-29T11:09:19.529 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.584 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.121) 0:04:11.811 ******* 2019-11-29T11:09:19.625 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.657 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.077) 0:04:11.888 ******* 2019-11-29T11:09:19.696 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-11-29T11:09:19.698 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:19.751 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:19 +0000 (0:00:00.093) 0:04:11.982 ******* 2019-11-29T11:09:20.286 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-11-29T11:09:20.288 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-11-29T11:09:20.323 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:20 +0000 (0:00:00.572) 0:04:12.554 ******* 2019-11-29T11:09:20.380 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-11-29T11:09:20.413 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:20 +0000 (0:00:00.090) 0:04:12.644 ******* 2019-11-29T11:09:21.102 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-11-29T11:09:21.104 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:21.585 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:21.605 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:22.071 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:22.072 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:22.537 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:22.562 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:23.026 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:23.029 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-29T11:09:23.081 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:23 +0000 (0:00:02.660) 0:04:15.305 ******* 2019-11-29T11:09:23.113 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:23 +0000 (0:00:00.039) 0:04:15.344 ******* 2019-11-29T11:09:23.403 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-29T11:09:23.408 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-11-29T11:09:23.464 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:23 +0000 (0:00:00.346) 0:04:15.691 ******* 2019-11-29T11:09:23.467 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-11-29T11:09:23.515 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:23 +0000 (0:00:00.046) 0:04:15.738 ******* TASK [testnode : Write /scratch_devs] ****************************************** skipping: [smithi069.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:23.550 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:23.825 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:23.878 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.111 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.165 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.409 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.476 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.739 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.788 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:24 +0000 (0:00:01.281) 0:04:17.019 ******* 2019-11-29T11:09:24.797 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-29T11:09:24.851 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:24 +0000 (0:00:00.062) 0:04:17.082 ******* 2019-11-29T11:09:25.112 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:09:25.152 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:25 +0000 (0:00:00.301) 0:04:17.383 ******* 2019-11-29T11:09:25.194 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-29T11:09:25.195 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-11-29T11:09:25.238 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:25 +0000 (0:00:00.085) 0:04:17.469 ******* 2019-11-29T11:09:25.273 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:25 +0000 (0:00:00.034) 0:04:17.504 ******* 2019-11-29T11:09:25.320 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-11-29T11:09:25.363 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:25 +0000 (0:00:00.090) 0:04:17.595 ******* 2019-11-29T11:09:25.947 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi069.front.sepia.ceph.com] => {"changed": true} 2019-11-29T11:09:25.995 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:25 +0000 (0:00:00.631) 0:04:18.226 ******* 2019-11-29T11:09:26.755 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-11-29T11:09:26.756 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-11-29T11:09:26.798 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:26 +0000 (0:00:00.802) 0:04:19.029 ******* 2019-11-29T11:09:26.825 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-11-29T11:09:26.827 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:26.869 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:26 +0000 (0:00:00.068) 0:04:19.098 ******* 2019-11-29T11:09:26.894 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-11-29T11:09:26.900 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:26.932 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:26 +0000 (0:00:00.065) 0:04:19.163 ******* 2019-11-29T11:09:26.959 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi163.front.sepia.ceph.com] => {"changed": true} 2019-11-29T11:09:27.007 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.075) 0:04:19.238 ******* 2019-11-29T11:09:27.013 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-11-29T11:09:27.014 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.054 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.046) 0:04:19.285 ******* 2019-11-29T11:09:27.080 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-11-29T11:09:27.082 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.117 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.064) 0:04:19.349 ******* 2019-11-29T11:09:27.145 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-11-29T11:09:27.150 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.186 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.068) 0:04:19.417 ******* 2019-11-29T11:09:27.214 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.252 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.066) 0:04:19.483 ******* 2019-11-29T11:09:27.280 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-11-29T11:09:27.282 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.320 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.067) 0:04:19.551 ******* 2019-11-29T11:09:27.803 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-11-29T11:09:27.845 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.525) 0:04:20.076 ******* 2019-11-29T11:09:27.873 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-11-29T11:09:27.874 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.912 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.066) 0:04:20.143 ******* 2019-11-29T11:09:27.939 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-11-29T11:09:27.941 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:27.980 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:27 +0000 (0:00:00.068) 0:04:20.212 ******* 2019-11-29T11:09:28.452 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-11-29T11:09:28.454 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-11-29T11:09:28.506 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.524) 0:04:20.736 ******* 2019-11-29T11:09:28.533 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.588 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.082) 0:04:20.819 ******* 2019-11-29T11:09:28.618 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-11-29T11:09:28.619 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.679 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.091) 0:04:20.911 ******* 2019-11-29T11:09:28.706 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-11-29T11:09:28.708 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.743 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.063) 0:04:20.974 ******* 2019-11-29T11:09:28.771 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-11-29T11:09:28.773 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.836 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.074) 0:04:21.049 ******* 2019-11-29T11:09:28.844 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.893 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.075) 0:04:21.125 ******* 2019-11-29T11:09:28.899 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-11-29T11:09:28.900 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-11-29T11:09:28.939 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.045) 0:04:21.170 ******* 2019-11-29T11:09:28.945 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-11-29T11:09:28.947 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:28.992 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:28 +0000 (0:00:00.046) 0:04:21.217 ******* TASK [testnode : Create the chrony.conf file] ********************************** 2019-11-29T11:09:28.997 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:29.036 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.050) 0:04:21.267 ******* 2019-11-29T11:09:29.043 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-11-29T11:09:29.044 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:29.084 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.047) 0:04:21.315 ******* 2019-11-29T11:09:29.548 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-11-29T11:09:29.552 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "tmp.mount syslog.target -.mount ntpdate.service sntp.service basic.target system.slice systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-11-29T11:09:29.573 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-11-29T11:09:29.574 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-29T11:09:29.592 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.508) 0:04:21.824 ******* 2019-11-29T11:09:29.599 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-11-29T11:09:29.600 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-11-29T11:09:29.640 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.047) 0:04:21.871 ******* 2019-11-29T11:09:29.666 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:29.667 INFO:teuthology.task.ansible.out:TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:29.705 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.064) 0:04:21.936 ******* 2019-11-29T11:09:29.732 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-11-29T11:09:29.733 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:29.772 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:29 +0000 (0:00:00.067) 0:04:22.003 ******* 2019-11-29T11:09:30.551 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-11-29T11:09:30.597 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:30 +0000 (0:00:00.823) 0:04:22.827 ******* 2019-11-29T11:09:30.627 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:30.661 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:30 +0000 (0:00:00.065) 0:04:22.892 ******* 2019-11-29T11:09:30.709 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-11-29T11:09:30.712 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-11-29T11:09:30.753 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:30 +0000 (0:00:00.087) 0:04:22.980 ******* 2019-11-29T11:09:30.782 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-11-29T11:09:30.783 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:30.820 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:30 +0000 (0:00:00.071) 0:04:23.051 ******* 2019-11-29T11:09:31.067 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-11-29T11:09:31.069 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-11-29T11:09:31.143 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:31 +0000 (0:00:00.322) 0:04:23.374 ******* 2019-11-29T11:09:31.329 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-11-29T11:09:31.330 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice sntp.service basic.target -.mount systemd-journald.socket syslog.target tmp.mount ntpdate.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target shutdown.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-11-29T11:09:31.332 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-11-29T11:09:31.334 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-29T11:09:31.371 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:31 +0000 (0:00:00.227) 0:04:23.601 ******* 2019-11-29T11:09:32.084 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-11-29T11:09:32.086 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-11-29T11:09:32.122 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:32 +0000 (0:00:00.751) 0:04:24.353 ******* 2019-11-29T11:09:32.342 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-11-29T11:09:32.348 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-11-29T11:09:32.380 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:32 +0000 (0:00:00.257) 0:04:24.610 ******* 2019-11-29T11:09:32.406 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-11-29T11:09:32.408 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:32.444 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:32 +0000 (0:00:00.065) 0:04:24.676 ******* 2019-11-29T11:09:32.760 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-11-29T11:09:32.762 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-11-29T11:09:32.807 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:32 +0000 (0:00:00.362) 0:04:25.038 ******* 2019-11-29T11:09:33.770 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-11-29T11:09:33.810 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:33 +0000 (0:00:01.003) 0:04:26.041 ******* 2019-11-29T11:09:33.811 INFO:teuthology.task.ansible.out: 2019-11-29T11:09:33.971 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-11-29T11:09:34.021 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.210) 0:04:26.252 ******* 2019-11-29T11:09:34.048 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:34.086 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.065) 0:04:26.318 ******* 2019-11-29T11:09:34.117 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-11-29T11:09:34.119 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-11-29T11:09:34.161 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.074) 0:04:26.392 ******* 2019-11-29T11:09:34.457 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-11-29T11:09:34.459 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-11-29T11:09:34.520 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.348) 0:04:26.740 ******* 2019-11-29T11:09:34.548 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-11-29T11:09:34.549 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:34.616 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.107) 0:04:26.848 ******* 2019-11-29T11:09:34.656 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-11-29T11:09:34.658 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-11-29T11:09:34.708 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.091) 0:04:26.939 ******* 2019-11-29T11:09:34.735 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-11-29T11:09:34.737 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:34.772 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:34 +0000 (0:00:00.064) 0:04:27.003 ******* 2019-11-29T11:09:35.099 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-11-29T11:09:35.168 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.395) 0:04:27.399 ******* 2019-11-29T11:09:35.272 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-11-29T11:09:35.273 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-11-29T11:09:35.328 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.156) 0:04:27.555 ******* 2019-11-29T11:09:35.351 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-11-29T11:09:35.353 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.396 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.070) 0:04:27.626 ******* 2019-11-29T11:09:35.422 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-11-29T11:09:35.468 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.073) 0:04:27.699 ******* 2019-11-29T11:09:35.497 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi069.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-11-29T11:09:35.540 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.069) 0:04:27.769 ******* 2019-11-29T11:09:35.565 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.604 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.065) 0:04:27.834 ******* 2019-11-29T11:09:35.663 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-11-29T11:09:35.668 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.676 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.688 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.727 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.123) 0:04:27.958 ******* 2019-11-29T11:09:35.765 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-11-29T11:09:35.767 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.777 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-11-29T11:09:35.781 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi069.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.842 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.111) 0:04:28.069 ******* 2019-11-29T11:09:35.850 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi069.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.897 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.059) 0:04:28.128 ******* 2019-11-29T11:09:35.903 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-11-29T11:09:35.904 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.946 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.049) 0:04:28.177 ******* 2019-11-29T11:09:35.954 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:35.993 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:35 +0000 (0:00:00.046) 0:04:28.224 ******* 2019-11-29T11:09:36.001 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi163.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-11-29T11:09:36.046 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.052) 0:04:28.277 ******* 2019-11-29T11:09:36.052 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-11-29T11:09:36.052 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.093 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.047) 0:04:28.325 ******* 2019-11-29T11:09:36.100 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.142 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.048) 0:04:28.373 ******* 2019-11-29T11:09:36.152 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.191 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.048) 0:04:28.422 ******* 2019-11-29T11:09:36.219 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-11-29T11:09:36.220 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.256 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.065) 0:04:28.488 ******* 2019-11-29T11:09:36.312 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.346 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.089) 0:04:28.577 ******* 2019-11-29T11:09:36.373 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-11-29T11:09:36.374 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.420 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.072) 0:04:28.650 ******* 2019-11-29T11:09:36.446 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-11-29T11:09:36.447 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.490 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.071) 0:04:28.722 ******* 2019-11-29T11:09:36.498 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-11-29T11:09:36.500 INFO:teuthology.task.ansible.out:changed: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-11-29T11:09:36.543 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.052) 0:04:28.774 ******* 2019-11-29T11:09:36.552 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-11-29T11:09:36.554 INFO:teuthology.task.ansible.out:skipping: [smithi069.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.602 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.058) 0:04:28.833 ******* 2019-11-29T11:09:36.876 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi163.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-11-29T11:09:36.910 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.305) 0:04:29.139 ******* 2019-11-29T11:09:36.944 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-11-29T11:09:36.946 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:36.982 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:36 +0000 (0:00:00.074) 0:04:29.213 ******* 2019-11-29T11:09:36.988 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-11-29T11:09:36.990 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.012462", "end": "2019-11-29 11:09:36.892776", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-11-29 11:09:36.880314", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:37.011 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-11-29T11:09:37.013 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.026 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.034 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.035 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.079 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.097) 0:04:29.310 ******* 2019-11-29T11:09:37.118 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi163.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.130 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi163.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.143 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.194 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.113) 0:04:29.423 ******* 2019-11-29T11:09:37.223 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-29T11:09:37.225 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.270 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.077) 0:04:29.501 ******* 2019-11-29T11:09:37.274 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-11-29T11:09:37.275 INFO:teuthology.task.ansible.out:ok: [smithi069.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006306", "end": "2019-11-29 11:09:37.208520", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-11-29 11:09:37.202214", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:37.288 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.326 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.057) 0:04:29.558 ******* 2019-11-29T11:09:37.353 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-11-29T11:09:37.355 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.398 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.071) 0:04:29.629 ******* 2019-11-29T11:09:37.424 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-11-29T11:09:37.426 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.464 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.066) 0:04:29.695 ******* 2019-11-29T11:09:37.495 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.538 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.073) 0:04:29.769 ******* 2019-11-29T11:09:37.560 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi069.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006713", "end": "2019-11-29 11:09:37.518086", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-11-29 11:09:37.511373", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:37.570 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.609 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.071) 0:04:29.840 ******* 2019-11-29T11:09:37.653 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-11-29T11:09:37.655 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.692 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.083) 0:04:29.923 ******* 2019-11-29T11:09:37.729 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-11-29T11:09:37.731 INFO:teuthology.task.ansible.out:skipping: [smithi163.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-29T11:09:37.782 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.084) 0:04:30.008 ******* 2019-11-29T11:09:37.920 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi069.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.034331", "end": "2019-11-29 11:09:37.860768", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-11-29 11:09:37.826437", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:37.984 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:37 +0000 (0:00:00.200) 0:04:30.209 ******* 2019-11-29T11:09:38.006 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-11-29T11:09:38.051 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:38 +0000 (0:00:00.073) 0:04:30.282 ******* 2019-11-29T11:09:38.110 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-11-29T11:09:38.112 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016259", "end": "2019-11-29 11:09:38.073194", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-11-29 11:09:38.056935", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:38.355 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-11-29T11:09:38.356 INFO:teuthology.task.ansible.out:changed: [smithi069.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-29T11:09:38.444 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi163.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007278", "end": "2019-11-29 11:09:38.401851", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-11-29 11:09:38.394573", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:38.750 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006670", "end": "2019-11-29 11:09:38.713528", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-11-29 11:09:38.706858", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:40.099 INFO:teuthology.task.ansible.out:ok: [smithi163.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.028921", "end": "2019-11-29 11:09:40.054195", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-11-29 11:09:39.025274", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-29T11:09:40.136 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:40 +0000 (0:00:02.085) 0:04:32.368 ******* 2019-11-29T11:09:40.166 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-11-29T11:09:40.208 INFO:teuthology.task.ansible.out:Friday 29 November 2019 11:09:40 +0000 (0:00:00.064) 0:04:32.433 ******* 2019-11-29T11:09:40.528 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi163.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-29T11:09:40.635 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-11-29T11:09:40.637 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-11-29T11:09:40.640 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi069.front.sepia.ceph.com : ok=121 changed=39 unreachable=0 failed=0 skipped=159 rescued=0 ignored=3 smithi163.front.sepia.ceph.com : ok=121 changed=39 unreachable=0 failed=0 skipped=159 rescued=0 ignored=3 Friday 29 November 2019 11:09:40 +0000 (0:00:00.438) 0:04:32.871 ******* =============================================================================== 2019-11-29T11:09:40.644 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 39.72s users : Update authorized_keys using the keys repo --------------------- 38.44s testnode : Install packages -------------------------------------------- 17.62s common : Update yum cache ---------------------------------------------- 12.07s testnode : Ensure ceph packages are not present. ----------------------- 10.35s users : Remove revoked users -------------------------------------------- 8.75s testnode : Zap all non-root disks --------------------------------------- 8.69s common : nrpe - Remove SELinux policy package --------------------------- 8.16s common : nrpe - Load SELinux policy package ----------------------------- 7.93s Gathering Facts --------------------------------------------------------- 3.46s common : Make sure smartmontools is installed --------------------------- 3.07s testnode : Ensure ceph dependency packages are not present. ------------- 2.97s ansible-managed : Add authorized keys for the ansible user. ------------- 2.80s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.72s testnode : Create logical volume(s) ------------------------------------- 2.66s users : Update authorized_keys for each user with literal keys ---------- 2.66s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.64s common : nrpe - Ensure SELinux policy is up to date --------------------- 2.42s testnode : Rebuild rpmdb ------------------------------------------------ 2.40s common : Configure local epel mirrorlists ------------------------------- 1.99s 2019-11-29T11:09:40.924 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2019-11-29T11:09:40.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:09:40.926 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi069.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:09:41.104 INFO:teuthology.orchestra.run.smithi069:> true 2019-11-29T11:09:41.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-29T11:09:41.500 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi163.front.sepia.ceph.com', 'timeout': 60} 2019-11-29T11:09:41.600 INFO:teuthology.orchestra.run.smithi163:> true 2019-11-29T11:09:41.969 INFO:teuthology.run_tasks:Running task clock... 2019-11-29T11:09:41.978 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-11-29T11:09:41.978 INFO:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-11-29T11:09:42.129 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-11-29T11:09:52.214 INFO:teuthology.orchestra.run.smithi069.stdout:ntpd: time slew -0.003746s 2019-11-29T11:09:52.252 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit not found. 2019-11-29T11:09:52.342 INFO:teuthology.orchestra.run.smithi069.stdout: remote refid st t when poll reach delay offset jitter 2019-11-29T11:09:52.342 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================== 2019-11-29T11:09:52.342 INFO:teuthology.orchestra.run.smithi069.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:09:52.343 INFO:teuthology.orchestra.run.smithi069.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:09:52.343 INFO:teuthology.orchestra.run.smithi069.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:09:52.343 INFO:teuthology.orchestra.run.smithi069.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:09:52.344 INFO:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-11-29T11:09:52.484 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-11-29T11:10:02.573 INFO:teuthology.orchestra.run.smithi163.stdout:ntpd: time slew -0.000900s 2019-11-29T11:10:02.614 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit not found. 2019-11-29T11:10:02.694 INFO:teuthology.orchestra.run.smithi163.stdout: remote refid st t when poll reach delay offset jitter 2019-11-29T11:10:02.695 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================== 2019-11-29T11:10:02.695 INFO:teuthology.orchestra.run.smithi163.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:10:02.695 INFO:teuthology.orchestra.run.smithi163.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:10:02.695 INFO:teuthology.orchestra.run.smithi163.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:10:02.695 INFO:teuthology.orchestra.run.smithi163.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-29T11:10:02.696 INFO:teuthology.run_tasks:Running task install... 2019-11-29T11:10:02.721 DEBUG:teuthology.task.install:project ceph 2019-11-29T11:10:02.722 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64'}} 2019-11-29T11:10:02.722 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64'} 2019-11-29T11:10:02.722 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-11-29T11:10:02.726 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-11-29T11:10:02.726 INFO:teuthology.task.install:extra packages: [] 2019-11-29T11:10:02.727 INFO:teuthology.orchestra.run.smithi069:> sudo lsb_release -is 2019-11-29T11:10:02.901 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS 2019-11-29T11:10:02.903 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:10:02.904 INFO:teuthology.orchestra.run.smithi163:> sudo lsb_release -is 2019-11-29T11:10:02.904 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-11-29T11:10:02.904 INFO:teuthology.orchestra.run.smithi069:> sudo lsb_release -is 2019-11-29T11:10:03.044 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS 2019-11-29T11:10:03.047 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:10:03.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:10:03.072 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS 2019-11-29T11:10:03.083 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:10:03.083 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-11-29T11:10:03.083 INFO:teuthology.orchestra.run.smithi163:> sudo lsb_release -is 2019-11-29T11:10:03.213 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS 2019-11-29T11:10:03.215 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:10:03.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:10:03.242 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuval-master/656c8e8049c2c1acd363143c842c2edf1fe09b64/centos/7/flavors/notcmalloc/ 2019-11-29T11:10:03.242 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7809.g656c8e8 2019-11-29T11:10:03.373 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-11-29T11:10:03.376 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuval-master/656c8e8049c2c1acd363143c842c2edf1fe09b64/centos/7/flavors/notcmalloc/ 2019-11-29T11:10:03.376 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7809.g656c8e8 2019-11-29T11:10:03.497 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-11-29T11:10:03.499 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-11-29T11:10:03.499 INFO:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/656c8e8049c2c1acd363143c842c2edf1fe09b64/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-11-29T11:10:03.618 INFO:teuthology.orchestra.run.smithi069:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-11-29T11:10:03.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-11-29T11:10:03.624 INFO:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/656c8e8049c2c1acd363143c842c2edf1fe09b64/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-11-29T11:10:03.706 INFO:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-11-29T11:10:03.725 INFO:teuthology.orchestra.run.smithi163:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-11-29T11:10:03.779 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2019-11-29T11:10:03.781 INFO:teuthology.orchestra.run.smithi069:> sudo yum clean all 2019-11-29T11:10:03.834 INFO:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-11-29T11:10:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2019-11-29T11:10:03.953 INFO:teuthology.orchestra.run.smithi163:> sudo yum clean all 2019-11-29T11:10:03.960 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:03.998 INFO:teuthology.orchestra.run.smithi069.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-11-29T11:10:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: : lab-extras updates 2019-11-29T11:10:04.042 INFO:teuthology.orchestra.run.smithi069.stdout:Cleaning up list of fastest mirrors 2019-11-29T11:10:04.043 INFO:teuthology.orchestra.run.smithi069.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-11-29T11:10:04.056 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw 2019-11-29T11:10:04.152 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:04.191 INFO:teuthology.orchestra.run.smithi163.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-11-29T11:10:04.191 INFO:teuthology.orchestra.run.smithi163.stdout: : lab-extras updates 2019-11-29T11:10:04.235 INFO:teuthology.orchestra.run.smithi163.stdout:Cleaning up list of fastest mirrors 2019-11-29T11:10:04.236 INFO:teuthology.orchestra.run.smithi163.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-11-29T11:10:04.248 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw 2019-11-29T11:10:04.249 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:04.286 INFO:teuthology.orchestra.run.smithi069.stdout:Determining fastest mirrors 2019-11-29T11:10:04.472 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:04.515 INFO:teuthology.orchestra.run.smithi163.stdout:Determining fastest mirrors 2019-11-29T11:10:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:10:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:05.100 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:10:05.101 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:05.101 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:05.101 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:12.424 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:12.484 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:12.779 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:10:12.780 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:12.780 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.792 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.835 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:10:12.835 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:12.835 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.846 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.899 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.900 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.904 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.907 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.908 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:12.908 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-base.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.916 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.918 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.919 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.920 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.921 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.922 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.925 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-11-29T11:10:12.927 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-11-29T11:10:12.927 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-11-29T11:10:12.927 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librados2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be an update 2019-11-29T11:10:12.928 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-11-29T11:10:12.929 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librgw2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.930 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:12.930 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.934 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.935 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.936 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.962 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.963 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.964 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librbd1.x86_64 2:15.0.0-7809.g656c8e8.el7 will be an update 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-29T11:10:12.965 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-11-29T11:10:12.966 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-rados.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-rbd.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-rgw.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-11-29T11:10:12.968 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.968 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.968 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.968 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:12.968 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-base.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.973 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.974 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.975 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.976 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.978 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:12.978 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.982 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-11-29T11:10:12.982 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-11-29T11:10:12.982 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-11-29T11:10:12.983 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librados2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be an update 2019-11-29T11:10:12.984 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-11-29T11:10:12.985 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librgw2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:12.988 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:12.988 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.003 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.003 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.003 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.004 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.005 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.005 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librbd1.x86_64 2:15.0.0-7809.g656c8e8.el7 will be an update 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-29T11:10:13.006 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-29T11:10:13.007 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:13.007 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-11-29T11:10:13.008 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.008 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.008 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-rados.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-rbd.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-rgw.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:13.009 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-11-29T11:10:13.819 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:13.868 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:13.905 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:13.905 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:10:13.908 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:13.908 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 6.2 M 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:10:13.909 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 5.6 M 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 19 M 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 20 k 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 490 k 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-11-29T11:10:13.910 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 374 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 5.4 M 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-argparse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 35 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-common x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 42 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: python-cephfs x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 102 k 2019-11-29T11:10:13.911 INFO:teuthology.orchestra.run.smithi163.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: python-rados x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 209 k 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: python-rbd x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 199 k 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: python-rgw x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 77 k 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout:Updating for dependencies: 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 3.5 M 2019-11-29T11:10:13.912 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.8 M 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+20 Dependent packages) 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade ( 2 Dependent packages) 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:13.913 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 45 M 2019-11-29T11:10:13.914 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:10:13.914 INFO:teuthology.orchestra.run.smithi163.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-29T11:10:13.946 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:13.946 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:10:13.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:10:13.950 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 6.2 M 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 5.6 M 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 19 M 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 20 k 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 490 k 2019-11-29T11:10:13.951 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 374 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 5.4 M 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-argparse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 35 k 2019-11-29T11:10:13.952 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-common x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 42 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: python-cephfs x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 102 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: python-rados x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 209 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: python-rbd x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 199 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: python-rgw x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 77 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-11-29T11:10:13.953 INFO:teuthology.orchestra.run.smithi069.stdout:Updating for dependencies: 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 3.5 M 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.8 M 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+20 Dependent packages) 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade ( 2 Dependent packages) 2019-11-29T11:10:13.954 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:13.955 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 45 M 2019-11-29T11:10:13.955 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:10:13.955 INFO:teuthology.orchestra.run.smithi069.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-29T11:10:17.324 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:10:17.325 INFO:teuthology.orchestra.run.smithi163.stdout:Total 13 MB/s | 45 MB 00:03 2019-11-29T11:10:17.341 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:10:17.412 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:10:17.677 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:10:17.678 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:10:18.364 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-29T11:10:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-11-29T11:10:19.062 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-11-29T11:10:19.348 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-ceph-argparse-15.0.0-7809.g656c8e8.el7.x86_64 4/25 2019-11-29T11:10:19.645 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 5/25 2019-11-29T11:10:20.026 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-11-29T11:10:20.709 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:10:20.709 INFO:teuthology.orchestra.run.smithi069.stdout:Total 6.6 MB/s | 45 MB 00:06 2019-11-29T11:10:20.723 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:10:20.792 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:10:20.792 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-11-29T11:10:21.052 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:10:21.053 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:10:21.178 INFO:teuthology.orchestra.run.smithi163.stdout: Updating : 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 8/25 2019-11-29T11:10:21.709 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-rados-15.0.0-7809.g656c8e8.el7.x86_64 9/25 2019-11-29T11:10:21.740 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-29T11:10:22.119 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-11-29T11:10:22.487 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-11-29T11:10:22.587 INFO:teuthology.orchestra.run.smithi163.stdout: Updating : 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 10/25 2019-11-29T11:10:22.810 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-ceph-argparse-15.0.0-7809.g656c8e8.el7.x86_64 4/25 2019-11-29T11:10:22.922 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:librgw2-15.0.0-7809.g656c8e8.el7.x86_64 11/25 2019-11-29T11:10:23.171 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 5/25 2019-11-29T11:10:23.406 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 12/25 2019-11-29T11:10:23.554 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-11-29T11:10:23.679 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-cephfs-15.0.0-7809.g656c8e8.el7.x86_64 13/25 2019-11-29T11:10:23.968 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-rgw-15.0.0-7809.g656c8e8.el7.x86_64 14/25 2019-11-29T11:10:24.303 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-rbd-15.0.0-7809.g656c8e8.el7.x86_64 15/25 2019-11-29T11:10:24.405 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-11-29T11:10:24.594 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:libradosstriper1-15.0.0-7809.g656c8e8.el7.x86_64 16/25 2019-11-29T11:10:24.784 INFO:teuthology.orchestra.run.smithi069.stdout: Updating : 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 8/25 2019-11-29T11:10:24.888 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-29T11:10:25.210 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 18/25 2019-11-29T11:10:25.339 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-rados-15.0.0-7809.g656c8e8.el7.x86_64 9/25 2019-11-29T11:10:26.269 INFO:teuthology.orchestra.run.smithi069.stdout: Updating : 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 10/25 2019-11-29T11:10:26.655 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:librgw2-15.0.0-7809.g656c8e8.el7.x86_64 11/25 2019-11-29T11:10:26.976 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 12/25 2019-11-29T11:10:27.282 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-cephfs-15.0.0-7809.g656c8e8.el7.x86_64 13/25 2019-11-29T11:10:27.564 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-rgw-15.0.0-7809.g656c8e8.el7.x86_64 14/25 2019-11-29T11:10:27.664 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 19/25 2019-11-29T11:10:27.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-rbd-15.0.0-7809.g656c8e8.el7.x86_64 15/25 2019-11-29T11:10:28.157 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:libradosstriper1-15.0.0-7809.g656c8e8.el7.x86_64 16/25 2019-11-29T11:10:28.452 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-29T11:10:28.774 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 18/25 2019-11-29T11:10:28.838 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 20/25 2019-11-29T11:10:29.398 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 21/25 2019-11-29T11:10:31.273 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 19/25 2019-11-29T11:10:32.431 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 20/25 2019-11-29T11:10:32.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 21/25 2019-11-29T11:10:39.506 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-selinux-15.0.0-7809.g656c8e8.el7.x86_64 22/25 2019-11-29T11:10:39.894 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 23/25 2019-11-29T11:10:40.145 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-29T11:10:40.474 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-29T11:10:40.599 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-29T11:10:40.722 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 2/25 2019-11-29T11:10:40.831 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:libradosstriper1-15.0.0-7809.g656c8e8.el7.x86_64 3/25 2019-11-29T11:10:40.907 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 4/25 2019-11-29T11:10:41.016 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7809.g656c8e8.el7.x86_64 5/25 2019-11-29T11:10:41.141 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 6/25 2019-11-29T11:10:41.242 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/25 2019-11-29T11:10:41.334 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 8/25 2019-11-29T11:10:41.435 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:librgw2-15.0.0-7809.g656c8e8.el7.x86_64 9/25 2019-11-29T11:10:41.535 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-rados-15.0.0-7809.g656c8e8.el7.x86_64 10/25 2019-11-29T11:10:41.636 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 11/25 2019-11-29T11:10:41.736 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/25 2019-11-29T11:10:41.870 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/25 2019-11-29T11:10:41.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 14/25 2019-11-29T11:10:42.104 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-rbd-15.0.0-7809.g656c8e8.el7.x86_64 15/25 2019-11-29T11:10:42.213 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 16/25 2019-11-29T11:10:42.322 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-29T11:10:42.414 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 18/25 2019-11-29T11:10:42.447 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-selinux-15.0.0-7809.g656c8e8.el7.x86_64 22/25 2019-11-29T11:10:42.548 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 19/25 2019-11-29T11:10:42.649 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-selinux-15.0.0-7809.g656c8e8.el7.x86_64 20/25 2019-11-29T11:10:42.733 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-11-29T11:10:42.825 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-cephfs-15.0.0-7809.g656c8e8.el7.x86_64 22/25 2019-11-29T11:10:42.869 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 23/25 2019-11-29T11:10:42.925 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-rgw-15.0.0-7809.g656c8e8.el7.x86_64 23/25 2019-11-29T11:10:42.927 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-29T11:10:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-29T11:10:43.429 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-29T11:10:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:10:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-11-29T11:10:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-argparse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-cephfs.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-rados.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: python-rbd.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: python-rgw.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Updated: 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: librados2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:43.539 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:10:43.669 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-test 2019-11-29T11:10:43.931 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:43.958 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-29T11:10:43.969 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:10:43.970 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:10:43.970 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:43.971 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:43.971 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 2/25 2019-11-29T11:10:44.218 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:libradosstriper1-15.0.0-7809.g656c8e8.el7.x86_64 3/25 2019-11-29T11:10:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 4/25 2019-11-29T11:10:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7809.g656c8e8.el7.x86_64 5/25 2019-11-29T11:10:44.586 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-radosgw-15.0.0-7809.g656c8e8.el7.x86_64 6/25 2019-11-29T11:10:44.613 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:44.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/25 2019-11-29T11:10:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 8/25 2019-11-29T11:10:44.929 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:librgw2-15.0.0-7809.g656c8e8.el7.x86_64 9/25 2019-11-29T11:10:44.939 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:10:44.940 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:44.940 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-test.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:44.957 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:44.987 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:44.988 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:44.988 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:44.989 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-11-29T11:10:44.989 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-11-29T11:10:44.991 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-11-29T11:10:44.992 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-11-29T11:10:44.994 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:44.995 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-11-29T11:10:45.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-rados-15.0.0-7809.g656c8e8.el7.x86_64 10/25 2019-11-29T11:10:45.130 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-base-15.0.0-7809.g656c8e8.el7.x86_64 11/25 2019-11-29T11:10:45.214 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/25 2019-11-29T11:10:45.331 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/25 2019-11-29T11:10:45.363 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:45.423 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 14/25 2019-11-29T11:10:45.524 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-rbd-15.0.0-7809.g656c8e8.el7.x86_64 15/25 2019-11-29T11:10:45.577 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:45.577 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:10:45.578 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:45.578 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:45.578 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 38 M 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-11-29T11:10:45.579 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:45.580 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 38 M 2019-11-29T11:10:45.581 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 134 M 2019-11-29T11:10:45.581 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:10:45.666 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-common-15.0.0-7809.g656c8e8.el7.x86_64 16/25 2019-11-29T11:10:45.767 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-29T11:10:45.892 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 18/25 2019-11-29T11:10:46.001 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 19/25 2019-11-29T11:10:46.110 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-selinux-15.0.0-7809.g656c8e8.el7.x86_64 20/25 2019-11-29T11:10:46.236 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-11-29T11:10:46.345 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-cephfs-15.0.0-7809.g656c8e8.el7.x86_64 22/25 2019-11-29T11:10:46.487 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-rgw-15.0.0-7809.g656c8e8.el7.x86_64 23/25 2019-11-29T11:10:46.488 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-29T11:10:46.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-29T11:10:46.853 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:46.853 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:10:46.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-11-29T11:10:46.854 INFO:teuthology.orchestra.run.smithi069.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.855 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-argparse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-common.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-cephfs.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-rados.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-rbd.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.856 INFO:teuthology.orchestra.run.smithi069.stdout: python-rgw.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Updated: 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout: librados2.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:46.857 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:10:46.981 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-test 2019-11-29T11:10:47.290 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:47.330 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:10:47.331 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:10:47.332 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:47.332 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:47.332 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:47.839 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:10:47.839 INFO:teuthology.orchestra.run.smithi163.stdout:Total 17 MB/s | 38 MB 00:02 2019-11-29T11:10:47.842 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:10:47.862 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:10:47.915 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:10:47.915 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:10:47.967 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:48.291 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:10:48.292 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:48.292 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-test.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:48.309 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:48.338 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:48.339 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:48.340 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:48.341 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-11-29T11:10:48.341 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-11-29T11:10:48.342 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-11-29T11:10:48.344 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-11-29T11:10:48.347 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:48.347 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-11-29T11:10:48.635 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-11-29T11:10:48.705 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:48.921 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-29T11:10:48.925 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:48.925 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:10:48.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:48.926 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 38 M 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:10:48.927 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:48.928 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:10:48.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:48.929 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 38 M 2019-11-29T11:10:48.929 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 134 M 2019-11-29T11:10:48.929 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:10:49.249 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-11-29T11:10:51.003 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:10:51.003 INFO:teuthology.orchestra.run.smithi069.stdout:Total 18 MB/s | 38 MB 00:02 2019-11-29T11:10:51.006 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:10:51.028 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:10:51.115 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:10:51.116 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:10:51.826 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-11-29T11:10:52.151 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-29T11:10:52.499 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-11-29T11:10:53.243 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-11-29T11:10:53.844 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 5/5 2019-11-29T11:10:53.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 1/5 2019-11-29T11:10:54.075 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-11-29T11:10:54.192 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-11-29T11:10:54.326 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-11-29T11:10:54.853 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-11-29T11:10:54.900 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:54.900 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:10:54.900 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:54.900 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:54.901 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:10:54.901 INFO:teuthology.orchestra.run.smithi163.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-11-29T11:10:54.901 INFO:teuthology.orchestra.run.smithi163.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-11-29T11:10:54.901 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:54.901 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:10:54.981 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph 2019-11-29T11:10:55.244 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:55.287 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:10:55.288 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:10:55.289 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:55.289 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:55.289 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:55.930 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:56.273 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:10:56.273 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.274 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:56.274 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.296 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.297 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.298 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.300 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.300 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:56.310 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:56.312 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.313 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.315 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.316 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.317 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.318 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:56.319 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:56.327 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:56.327 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.327 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-11-29T11:10:56.328 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-11-29T11:10:56.328 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-11-29T11:10:56.328 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:56.329 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:56.330 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:56.331 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:56.332 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:56.332 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-11-29T11:10:56.333 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-11-29T11:10:56.334 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-11-29T11:10:56.334 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-11-29T11:10:56.335 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-11-29T11:10:56.336 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.336 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-11-29T11:10:56.336 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-11-29T11:10:56.337 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-11-29T11:10:56.337 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-11-29T11:10:56.338 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-11-29T11:10:56.338 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-11-29T11:10:56.339 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-11-29T11:10:56.339 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-11-29T11:10:56.339 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-11-29T11:10:56.339 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.340 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-11-29T11:10:56.340 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-11-29T11:10:56.341 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-11-29T11:10:56.341 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.341 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-11-29T11:10:56.342 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-11-29T11:10:56.342 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:10:56.342 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-11-29T11:10:56.542 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-11-29T11:10:56.700 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:56.862 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 5/5 2019-11-29T11:10:56.925 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:56.925 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:10:56.929 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 2.0 k 2019-11-29T11:10:56.931 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:10:56.931 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.9 M 2019-11-29T11:10:56.932 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.7 M 2019-11-29T11:10:56.932 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 4.3 M 2019-11-29T11:10:56.932 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 16 M 2019-11-29T11:10:56.932 INFO:teuthology.orchestra.run.smithi163.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-11-29T11:10:56.932 INFO:teuthology.orchestra.run.smithi163.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-11-29T11:10:56.933 INFO:teuthology.orchestra.run.smithi163.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-11-29T11:10:56.933 INFO:teuthology.orchestra.run.smithi163.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-11-29T11:10:56.933 INFO:teuthology.orchestra.run.smithi163.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-11-29T11:10:56.933 INFO:teuthology.orchestra.run.smithi163.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-11-29T11:10:56.934 INFO:teuthology.orchestra.run.smithi163.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-11-29T11:10:56.934 INFO:teuthology.orchestra.run.smithi163.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-11-29T11:10:56.934 INFO:teuthology.orchestra.run.smithi163.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-11-29T11:10:56.935 INFO:teuthology.orchestra.run.smithi163.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+21 Dependent packages) 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:10:56.936 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 28 M 2019-11-29T11:10:56.937 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 87 M 2019-11-29T11:10:56.937 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:10:56.945 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-test-15.0.0-7809.g656c8e8.el7.x86_64 1/5 2019-11-29T11:10:57.053 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-11-29T11:10:57.138 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-11-29T11:10:57.221 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-11-29T11:10:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-11-29T11:10:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-11-29T11:10:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-11-29T11:10:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:57.726 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:10:57.851 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph 2019-11-29T11:10:58.101 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:10:58.136 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:10:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:10:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:10:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:10:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:10:58.452 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:10:58.453 INFO:teuthology.orchestra.run.smithi163.stdout:Total 18 MB/s | 28 MB 00:01 2019-11-29T11:10:58.469 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:10:58.501 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:10:58.786 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:10:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:10:58.862 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:10:59.111 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:10:59.111 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.112 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:59.113 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.142 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.143 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.145 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.146 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.146 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:59.156 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:59.158 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.159 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.161 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.162 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.162 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.164 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:10:59.165 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:59.167 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:10:59.171 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.172 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-11-29T11:10:59.172 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-11-29T11:10:59.173 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-11-29T11:10:59.174 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:59.175 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:59.176 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:59.176 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:59.177 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-11-29T11:10:59.178 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-11-29T11:10:59.178 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-11-29T11:10:59.179 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-11-29T11:10:59.180 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-11-29T11:10:59.180 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-11-29T11:10:59.181 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.181 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-11-29T11:10:59.181 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-11-29T11:10:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-11-29T11:10:59.183 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-11-29T11:10:59.183 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-11-29T11:10:59.183 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-11-29T11:10:59.184 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-11-29T11:10:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-11-29T11:10:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-11-29T11:10:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-11-29T11:10:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-11-29T11:10:59.188 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-11-29T11:10:59.188 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.188 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-11-29T11:10:59.188 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-11-29T11:10:59.189 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:10:59.189 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-11-29T11:10:59.547 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:10:59.766 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:59.766 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:59.769 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 2.0 k 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.9 M 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 1.7 M 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 4.3 M 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 16 M 2019-11-29T11:10:59.770 INFO:teuthology.orchestra.run.smithi069.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-11-29T11:10:59.771 INFO:teuthology.orchestra.run.smithi069.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-11-29T11:10:59.772 INFO:teuthology.orchestra.run.smithi069.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:10:59.773 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+21 Dependent packages) 2019-11-29T11:10:59.774 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:10:59.774 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 28 M 2019-11-29T11:10:59.774 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 87 M 2019-11-29T11:10:59.774 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:10:59.991 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-11-29T11:11:00.281 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-11-29T11:11:01.194 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:01.194 INFO:teuthology.orchestra.run.smithi069.stdout:Total 19 MB/s | 28 MB 00:01 2019-11-29T11:11:01.208 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:01.234 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:01.540 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:01.541 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:02.123 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-11-29T11:11:02.439 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-11-29T11:11:02.579 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-osd-15.0.0-7809.g656c8e8.el7.x86_64 4/22 2019-11-29T11:11:02.773 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-11-29T11:11:03.062 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 5/22 2019-11-29T11:11:03.653 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 6/22 2019-11-29T11:11:04.085 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 7/22 2019-11-29T11:11:04.607 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 8/22 2019-11-29T11:11:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-11-29T11:11:04.893 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 9/22 2019-11-29T11:11:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-osd-15.0.0-7809.g656c8e8.el7.x86_64 4/22 2019-11-29T11:11:05.410 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 10/22 2019-11-29T11:11:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 5/22 2019-11-29T11:11:05.781 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mds-15.0.0-7809.g656c8e8.el7.x86_64 11/22 2019-11-29T11:11:05.959 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 6/22 2019-11-29T11:11:06.018 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-six-1.9.0-0.el7.noarch 12/22 2019-11-29T11:11:06.348 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 13/22 2019-11-29T11:11:06.445 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 7/22 2019-11-29T11:11:06.711 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-11-29T11:11:06.944 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 8/22 2019-11-29T11:11:07.270 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 9/22 2019-11-29T11:11:07.506 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-11-29T11:11:07.812 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 10/22 2019-11-29T11:11:07.908 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mon-15.0.0-7809.g656c8e8.el7.x86_64 16/22 2019-11-29T11:11:08.264 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mds-15.0.0-7809.g656c8e8.el7.x86_64 11/22 2019-11-29T11:11:08.321 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 17/22 2019-11-29T11:11:08.505 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-six-1.9.0-0.el7.noarch 12/22 2019-11-29T11:11:08.655 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-mako-0.8.1-2.el7.noarch 18/22 2019-11-29T11:11:08.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 13/22 2019-11-29T11:11:09.071 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-11-29T11:11:09.236 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-11-29T11:11:09.734 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-11-29T11:11:10.027 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-11-29T11:11:10.152 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 21/22 2019-11-29T11:11:10.395 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mon-15.0.0-7809.g656c8e8.el7.x86_64 16/22 2019-11-29T11:11:10.457 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 22/22 2019-11-29T11:11:10.559 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-11-29T11:11:10.667 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 2/22 2019-11-29T11:11:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-11-29T11:11:10.812 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 17/22 2019-11-29T11:11:10.885 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-11-29T11:11:10.985 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mon-15.0.0-7809.g656c8e8.el7.x86_64 5/22 2019-11-29T11:11:11.111 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-11-29T11:11:11.159 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-mako-0.8.1-2.el7.noarch 18/22 2019-11-29T11:11:11.212 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 7/22 2019-11-29T11:11:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 8/22 2019-11-29T11:11:11.413 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 9/22 2019-11-29T11:11:11.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mds-15.0.0-7809.g656c8e8.el7.x86_64 10/22 2019-11-29T11:11:11.609 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-11-29T11:11:11.631 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 11/22 2019-11-29T11:11:11.732 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 12/22 2019-11-29T11:11:11.856 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 13/22 2019-11-29T11:11:11.973 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 14/22 2019-11-29T11:11:12.092 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-11-29T11:11:12.208 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-11-29T11:11:12.308 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-11-29T11:11:12.312 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-11-29T11:11:12.408 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 18/22 2019-11-29T11:11:12.559 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 19/22 2019-11-29T11:11:12.659 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 20/22 2019-11-29T11:11:12.760 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 21/22 2019-11-29T11:11:12.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 21/22 2019-11-29T11:11:13.050 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 22/22 2019-11-29T11:11:13.172 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-11-29T11:11:13.181 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-osd-15.0.0-7809.g656c8e8.el7.x86_64 22/22 2019-11-29T11:11:13.181 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:13.181 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:13.181 INFO:teuthology.orchestra.run.smithi163.stdout: ceph.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:13.182 INFO:teuthology.orchestra.run.smithi163.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-11-29T11:11:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-11-29T11:11:13.184 INFO:teuthology.orchestra.run.smithi163.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-11-29T11:11:13.185 INFO:teuthology.orchestra.run.smithi163.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-11-29T11:11:13.185 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:13.185 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:13.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 2/22 2019-11-29T11:11:13.309 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-daemon 2019-11-29T11:11:13.422 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-11-29T11:11:13.539 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-11-29T11:11:13.581 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:13.625 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:13.626 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:13.627 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:13.627 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:13.627 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:13.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mon-15.0.0-7809.g656c8e8.el7.x86_64 5/22 2019-11-29T11:11:13.787 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-11-29T11:11:14.241 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 7/22 2019-11-29T11:11:14.269 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:14.391 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 8/22 2019-11-29T11:11:14.494 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-15.0.0-7809.g656c8e8.el7.x86_64 9/22 2019-11-29T11:11:14.606 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:14.607 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:14.607 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:14.609 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mds-15.0.0-7809.g656c8e8.el7.x86_64 10/22 2019-11-29T11:11:14.701 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 11/22 2019-11-29T11:11:14.785 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 12/22 2019-11-29T11:11:14.904 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 13/22 2019-11-29T11:11:14.973 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:14.994 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 14/22 2019-11-29T11:11:15.136 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-11-29T11:11:15.200 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:15.200 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:15.201 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-daemon x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 17 k 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 17 k 2019-11-29T11:11:15.202 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 68 k 2019-11-29T11:11:15.203 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:15.245 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-11-29T11:11:15.345 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-11-29T11:11:15.479 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 18/22 2019-11-29T11:11:15.484 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:15.492 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:15.494 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:15.495 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:15.596 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 19/22 2019-11-29T11:11:15.730 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 20/22 2019-11-29T11:11:15.856 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 21/22 2019-11-29T11:11:16.305 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-daemon-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:11:16.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-osd-15.0.0-7809.g656c8e8.el7.x86_64 22/22 2019-11-29T11:11:16.376 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:16.377 INFO:teuthology.orchestra.run.smithi069.stdout: ceph.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:11:16.377 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.377 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.378 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.378 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.378 INFO:teuthology.orchestra.run.smithi069.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-11-29T11:11:16.378 INFO:teuthology.orchestra.run.smithi069.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-11-29T11:11:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-11-29T11:11:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-11-29T11:11:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-11-29T11:11:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-11-29T11:11:16.379 INFO:teuthology.orchestra.run.smithi069.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-11-29T11:11:16.380 INFO:teuthology.orchestra.run.smithi069.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-11-29T11:11:16.380 INFO:teuthology.orchestra.run.smithi069.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-11-29T11:11:16.381 INFO:teuthology.orchestra.run.smithi069.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-11-29T11:11:16.381 INFO:teuthology.orchestra.run.smithi069.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-11-29T11:11:16.381 INFO:teuthology.orchestra.run.smithi069.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-11-29T11:11:16.382 INFO:teuthology.orchestra.run.smithi069.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-11-29T11:11:16.382 INFO:teuthology.orchestra.run.smithi069.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-11-29T11:11:16.383 INFO:teuthology.orchestra.run.smithi069.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-11-29T11:11:16.383 INFO:teuthology.orchestra.run.smithi069.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-11-29T11:11:16.383 INFO:teuthology.orchestra.run.smithi069.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-11-29T11:11:16.383 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:16.502 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-daemon 2019-11-29T11:11:16.694 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-daemon-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:11:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:16.695 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-daemon.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:16.695 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:16.768 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:16.810 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:16.811 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:16.813 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:16.813 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:16.813 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:16.817 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr 2019-11-29T11:11:17.103 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:17.148 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:17.149 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:17.149 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:17.149 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:17.150 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:17.456 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:17.786 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:17.786 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:17.786 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:17.804 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:18.144 INFO:teuthology.orchestra.run.smithi163.stdout:Package 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:11:18.144 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do 2019-11-29T11:11:18.150 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:18.205 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-dashboard 2019-11-29T11:11:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:18.373 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:18.374 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:11:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:11:18.374 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:18.376 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-daemon x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 17 k 2019-11-29T11:11:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:18.377 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:18.378 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:18.378 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T11:11:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:18.378 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 17 k 2019-11-29T11:11:18.378 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 68 k 2019-11-29T11:11:18.379 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:18.442 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:18.484 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:18.485 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:18.486 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:18.486 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:18.487 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:18.686 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:18.697 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:18.698 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:18.698 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:19.157 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:19.489 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:19.490 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:19.490 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:19.493 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:19.522 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:19.523 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:19.524 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:19.524 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-11-29T11:11:19.524 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-11-29T11:11:19.525 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:19.525 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-11-29T11:11:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-daemon-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:11:19.890 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-daemon-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:11:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:20.080 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-daemon.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:20.080 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:20.128 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:20.128 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:20.131 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:20.131 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:20.131 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:11:20.131 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:20.131 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:20.132 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 3.9 M 2019-11-29T11:11:20.132 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:11:20.132 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 18 k 2019-11-29T11:11:20.132 INFO:teuthology.orchestra.run.smithi163.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-11-29T11:11:20.144 INFO:teuthology.orchestra.run.smithi163.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+3 Dependent packages) 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:20.145 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 4.6 M 2019-11-29T11:11:20.146 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 20 M 2019-11-29T11:11:20.148 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:20.214 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr 2019-11-29T11:11:20.404 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:20.447 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:20.449 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:20.449 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:20.451 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:20.452 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:20.734 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:20.734 INFO:teuthology.orchestra.run.smithi163.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-11-29T11:11:20.741 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:20.749 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:20.952 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:20.953 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:21.035 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:21.356 INFO:teuthology.orchestra.run.smithi069.stdout:Package 2:ceph-mgr-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:11:21.357 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do 2019-11-29T11:11:21.417 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-dashboard 2019-11-29T11:11:21.581 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7809.g656c8e8.el7.noarch 1/4 2019-11-29T11:11:21.635 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:21.679 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:21.680 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:21.681 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:21.681 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:21.681 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-11-29T11:11:22.325 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:22.656 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:22.657 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:22.657 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:22.660 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7809.g656c8e8.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:22.689 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:22.690 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:22.690 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:22.690 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-11-29T11:11:22.691 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-11-29T11:11:22.691 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:22.691 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-11-29T11:11:23.053 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:23.140 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-11-29T11:11:23.274 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:23.275 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:23.275 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 3.9 M 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:11:23.276 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 18 k 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+3 Dependent packages) 2019-11-29T11:11:23.277 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:23.278 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 4.6 M 2019-11-29T11:11:23.278 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 20 M 2019-11-29T11:11:23.278 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:23.892 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:23.893 INFO:teuthology.orchestra.run.smithi069.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-11-29T11:11:23.905 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:23.909 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:24.197 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:24.197 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:24.672 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 4/4 2019-11-29T11:11:24.822 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-11-29T11:11:24.944 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2/4 2019-11-29T11:11:25.025 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7809.g656c8e8.el7.noarch 1/4 2019-11-29T11:11:25.048 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7809.g656c8e8.el7.noarch 3/4 2019-11-29T11:11:25.455 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-11-29T11:11:25.534 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:25.535 INFO:teuthology.orchestra.run.smithi163.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-11-29T11:11:25.536 INFO:teuthology.orchestra.run.smithi163.stdout: python-routes.noarch 0:1.13-2.el7 2019-11-29T11:11:25.536 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:25.536 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:25.657 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-11-29T11:11:25.906 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:25.949 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:25.949 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:25.951 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:25.951 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:25.951 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:26.594 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:26.662 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-11-29T11:11:26.929 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:26.930 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:26.930 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:27.083 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 4/4 2019-11-29T11:11:27.212 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-11-29T11:11:27.293 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:27.300 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7809.g656c8e8.el7.noarch 2/4 2019-11-29T11:11:27.409 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7809.g656c8e8.el7.noarch 3/4 2019-11-29T11:11:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:27.521 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:27.522 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:11:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:27.523 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-cloud 2019-11-29T11:11:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 82 k 2019-11-29T11:11:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:27.524 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:27.524 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:27.525 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2019-11-29T11:11:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:27.525 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 82 k 2019-11-29T11:11:27.525 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 411 k 2019-11-29T11:11:27.525 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:27.822 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:27.831 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:27.863 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:27.863 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:27.962 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-11-29T11:11:27.963 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:27.963 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:27.964 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:27.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:27.964 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:11:27.964 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:27.965 INFO:teuthology.orchestra.run.smithi069.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-11-29T11:11:27.965 INFO:teuthology.orchestra.run.smithi069.stdout: python-routes.noarch 0:1.13-2.el7 2019-11-29T11:11:27.965 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:27.965 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:28.091 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-11-29T11:11:28.312 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:28.355 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:28.356 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:28.357 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:28.357 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:28.358 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:28.506 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7809.g656c8e8.el7.n 1/1 2019-11-29T11:11:29.000 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:29.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7809.g656c8e8.el7.n 1/1 2019-11-29T11:11:29.003 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:29.003 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:29.003 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:29.003 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:29.003 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:29.123 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-diskprediction-local 2019-11-29T11:11:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:29.382 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:29.425 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:29.427 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:29.428 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:29.428 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:29.428 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:29.695 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:29.920 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:29.920 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:29.920 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:29.920 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-cloud 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout: noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 82 k 2019-11-29T11:11:29.921 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:29.922 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:29.922 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:29.922 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T11:11:29.922 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:29.922 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 82 k 2019-11-29T11:11:29.923 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 411 k 2019-11-29T11:11:29.923 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:30.075 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:30.273 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:30.277 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:30.331 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:30.331 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:30.411 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:30.411 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:30.411 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:30.416 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:30.443 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:30.443 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-11-29T11:11:30.450 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:30.451 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:30.453 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:30.454 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:30.457 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-11-29T11:11:30.458 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-11-29T11:11:30.458 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-29T11:11:30.458 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-29T11:11:30.459 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:30.459 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-11-29T11:11:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:31.047 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:31.047 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:31.048 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:31.048 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:31.048 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:11:31.048 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout: noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 8.6 M 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-11-29T11:11:31.049 INFO:teuthology.orchestra.run.smithi163.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:31.050 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 19 M 2019-11-29T11:11:31.051 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 104 M 2019-11-29T11:11:31.051 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:31.059 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7809.g656c8e8.el7.n 1/1 2019-11-29T11:11:31.464 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7809.g656c8e8.el7.n 1/1 2019-11-29T11:11:31.465 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:31.465 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:31.465 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:31.465 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:31.465 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:31.626 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-diskprediction-local 2019-11-29T11:11:31.737 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:31.738 INFO:teuthology.orchestra.run.smithi163.stdout:Total 28 MB/s | 19 MB 00:00 2019-11-29T11:11:31.749 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:31.777 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:31.848 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:31.887 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:31.889 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:31.889 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:31.889 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:31.889 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:31.932 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:31.945 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:32.526 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:32.828 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-11-29T11:11:32.867 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:32.867 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:32.867 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:32.867 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:32.895 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:32.895 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-11-29T11:11:32.901 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:32.902 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:32.904 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-29T11:11:32.905 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:32.905 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-11-29T11:11:32.905 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-11-29T11:11:32.906 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-29T11:11:32.908 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-29T11:11:32.908 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:32.908 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-11-29T11:11:33.250 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-11-29T11:11:33.271 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:33.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:33.493 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:33.494 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:33.494 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:33.494 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout: noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 8.6 M 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-11-29T11:11:33.495 INFO:teuthology.orchestra.run.smithi069.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:11:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:33.497 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 19 M 2019-11-29T11:11:33.497 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 104 M 2019-11-29T11:11:33.497 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:34.381 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:34.381 INFO:teuthology.orchestra.run.smithi069.stdout:Total 22 MB/s | 19 MB 00:00 2019-11-29T11:11:34.390 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:34.417 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:34.614 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:34.614 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:35.056 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-29T11:11:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-11-29T11:11:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-11-29T11:11:36.749 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-29T11:11:37.029 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.n 5/5 2019-11-29T11:11:37.138 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-11-29T11:11:37.237 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.n 2/5 2019-11-29T11:11:37.367 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-29T11:11:37.467 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-29T11:11:37.737 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-29T11:11:37.961 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-11-29T11:11:37.961 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:37.961 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:37.961 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:37.961 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:37.962 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:11:37.962 INFO:teuthology.orchestra.run.smithi163.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-11-29T11:11:37.962 INFO:teuthology.orchestra.run.smithi163.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-11-29T11:11:37.962 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:37.962 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:38.086 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-rook 2019-11-29T11:11:38.362 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:38.408 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:38.409 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:38.409 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:38.409 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:38.410 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:39.078 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:39.421 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:39.421 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:39.421 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:39.423 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:39.453 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:39.453 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-11-29T11:11:39.455 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:39.456 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:39.457 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:39.458 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:39.459 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:39.459 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-11-29T11:11:39.459 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-11-29T11:11:39.460 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-11-29T11:11:39.460 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:39.461 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:39.462 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:39.463 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-11-29T11:11:39.463 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-11-29T11:11:39.464 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:39.464 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-11-29T11:11:39.465 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-11-29T11:11:39.465 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-11-29T11:11:39.466 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-11-29T11:11:39.466 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-11-29T11:11:39.466 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:39.466 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-11-29T11:11:39.475 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-29T11:11:39.832 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:39.907 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.n 5/5 2019-11-29T11:11:39.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-11-29T11:11:40.060 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:40.060 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:40.061 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:40.061 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:40.061 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 23 k 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-11-29T11:11:40.062 INFO:teuthology.orchestra.run.smithi163.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-11-29T11:11:40.063 INFO:teuthology.orchestra.run.smithi163.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-11-29T11:11:40.064 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:40.064 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:40.064 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:40.064 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+10 Dependent packages) 2019-11-29T11:11:40.064 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:40.065 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 1.6 M 2019-11-29T11:11:40.065 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 24 M 2019-11-29T11:11:40.065 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:40.071 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7809.g656c8e8.el7.n 2/5 2019-11-29T11:11:40.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-29T11:11:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-29T11:11:40.773 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-11-29T11:11:40.773 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-11-29T11:11:40.774 INFO:teuthology.orchestra.run.smithi069.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-11-29T11:11:40.775 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:40.775 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:40.879 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:40.880 INFO:teuthology.orchestra.run.smithi163.stdout:Total 2.0 MB/s | 1.6 MB 00:00 2019-11-29T11:11:40.891 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-rook 2019-11-29T11:11:40.895 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:40.908 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:41.037 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:41.037 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:41.099 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:41.139 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:41.762 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-11-29T11:11:41.780 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:42.101 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-11-29T11:11:42.104 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:42.104 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:42.104 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:42.105 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-11-29T11:11:42.138 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:42.138 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:42.139 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:42.140 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:42.149 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-11-29T11:11:42.150 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-11-29T11:11:42.151 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-11-29T11:11:42.151 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-11-29T11:11:42.151 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-11-29T11:11:42.151 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:42.152 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-11-29T11:11:42.479 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-11-29T11:11:42.511 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:42.733 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:42.733 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:42.734 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:42.734 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:42.735 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:11:42.735 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:42.735 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:42.735 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 23 k 2019-11-29T11:11:42.735 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-11-29T11:11:42.736 INFO:teuthology.orchestra.run.smithi069.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+10 Dependent packages) 2019-11-29T11:11:42.737 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:42.738 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 1.6 M 2019-11-29T11:11:42.738 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 24 M 2019-11-29T11:11:42.738 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:42.760 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-11-29T11:11:43.124 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-11-29T11:11:43.461 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-11-29T11:11:43.599 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:43.599 INFO:teuthology.orchestra.run.smithi069.stdout:Total 1.9 MB/s | 1.6 MB 00:00 2019-11-29T11:11:43.613 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:43.625 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:43.722 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:43.723 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:43.788 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-11-29T11:11:44.110 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-11-29T11:11:44.769 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-11-29T11:11:45.178 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-11-29T11:11:45.489 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-11-29T11:11:45.544 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-11-29T11:11:45.831 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-11-29T11:11:45.897 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-11-29T11:11:46.221 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-11-29T11:11:46.222 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 11/11 2019-11-29T11:11:46.306 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-11-29T11:11:46.390 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-11-29T11:11:46.496 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-11-29T11:11:46.592 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-11-29T11:11:46.596 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-29T11:11:46.705 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-11-29T11:11:46.822 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-11-29T11:11:46.931 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-11-29T11:11:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-11-29T11:11:47.040 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-11-29T11:11:47.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-11-29T11:11:47.257 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 10/11 2019-11-29T11:11:47.368 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-11-29T11:11:47.738 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 11/11 2019-11-29T11:11:47.738 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:47.738 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-11-29T11:11:47.739 INFO:teuthology.orchestra.run.smithi163.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-11-29T11:11:47.740 INFO:teuthology.orchestra.run.smithi163.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-11-29T11:11:47.740 INFO:teuthology.orchestra.run.smithi163.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-11-29T11:11:47.740 INFO:teuthology.orchestra.run.smithi163.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-11-29T11:11:47.740 INFO:teuthology.orchestra.run.smithi163.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-11-29T11:11:47.742 INFO:teuthology.orchestra.run.smithi163.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-11-29T11:11:47.742 INFO:teuthology.orchestra.run.smithi163.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-11-29T11:11:47.742 INFO:teuthology.orchestra.run.smithi163.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-11-29T11:11:47.743 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:47.743 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:47.855 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-ssh 2019-11-29T11:11:48.115 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:48.161 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:48.162 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:48.162 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:48.163 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:48.163 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:48.676 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-11-29T11:11:48.820 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:49.074 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-11-29T11:11:49.163 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:49.164 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:49.164 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:49.164 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:49.191 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:49.191 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-11-29T11:11:49.191 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-11-29T11:11:49.192 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:49.192 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-11-29T11:11:49.561 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:49.710 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 11/11 2019-11-29T11:11:49.790 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:49.790 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-ssh noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 33 k 2019-11-29T11:11:49.791 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:11:49.792 INFO:teuthology.orchestra.run.smithi163.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-11-29T11:11:49.792 INFO:teuthology.orchestra.run.smithi163.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-11-29T11:11:49.792 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:49.792 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+2 Dependent packages) 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 316 k 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 1.1 M 2019-11-29T11:11:49.793 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:49.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-11-29T11:11:49.950 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-11-29T11:11:50.059 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-11-29T11:11:50.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-29T11:11:50.293 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-11-29T11:11:50.329 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:50.330 INFO:teuthology.orchestra.run.smithi163.stdout:Total 587 kB/s | 316 kB 00:00 2019-11-29T11:11:50.333 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:50.342 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:50.405 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-11-29T11:11:50.421 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:50.421 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:50.478 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-11-29T11:11:50.562 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-11-29T11:11:50.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-11-29T11:11:50.771 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 10/11 2019-11-29T11:11:51.223 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-11-29T11:11:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7809.g656c8e8.el7.noarch 11/11 2019-11-29T11:11:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:51.246 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-11-29T11:11:51.247 INFO:teuthology.orchestra.run.smithi069.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:51.248 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:51.368 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-ssh 2019-11-29T11:11:51.458 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-11-29T11:11:51.550 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:51.587 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:51.588 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:51.589 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:51.589 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:51.589 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:51.711 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 3/3 2019-11-29T11:11:51.814 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-11-29T11:11:51.956 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-11-29T11:11:52.201 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:52.441 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 3/3 2019-11-29T11:11:52.441 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:52.441 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:11:52.441 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:52.441 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:52.442 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:11:52.442 INFO:teuthology.orchestra.run.smithi163.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-11-29T11:11:52.442 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:52.442 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:11:52.523 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:52.523 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:52.524 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:52.526 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 2019-11-29T11:11:52.554 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:52.555 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-11-29T11:11:52.556 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-11-29T11:11:52.558 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:52.558 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-11-29T11:11:52.573 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-fuse 2019-11-29T11:11:52.830 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:52.874 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:52.876 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:11:52.877 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:52.877 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:52.877 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:52.924 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:53.148 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:53.148 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:53.149 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:53.149 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:53.149 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-ssh noarch 2:15.0.0-7809.g656c8e8.el7 ceph-noarch 33 k 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-11-29T11:11:53.150 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+2 Dependent packages) 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 316 k 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 1.1 M 2019-11-29T11:11:53.151 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:53.519 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:53.690 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:53.691 INFO:teuthology.orchestra.run.smithi069.stdout:Total 584 kB/s | 316 kB 00:00 2019-11-29T11:11:53.694 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:53.702 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:53.785 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:53.785 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:53.857 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:11:53.857 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:53.858 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:53.867 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python36 for package: 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:11:53.899 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:11:53.900 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:53.900 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-11-29T11:11:53.901 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:53.903 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:53.904 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:53.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:53.905 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:11:53.905 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-11-29T11:11:53.911 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-11-29T11:11:53.911 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-11-29T11:11:54.275 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:54.514 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:54.514 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:11:54.514 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:54.515 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:54.515 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:11:54.515 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:11:54.515 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:54.516 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:11:54.516 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 566 k 2019-11-29T11:11:54.516 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:11:54.516 INFO:teuthology.orchestra.run.smithi163.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-11-29T11:11:54.517 INFO:teuthology.orchestra.run.smithi163.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-11-29T11:11:54.517 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-11-29T11:11:54.517 INFO:teuthology.orchestra.run.smithi163.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-11-29T11:11:54.518 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:54.518 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:11:54.518 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:11:54.518 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:11:54.519 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:11:54.520 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 10 M 2019-11-29T11:11:54.520 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 49 M 2019-11-29T11:11:54.520 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:11:54.693 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-11-29T11:11:54.957 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:54.957 INFO:teuthology.orchestra.run.smithi163.stdout:Total 22 MB/s | 10 MB 00:00 2019-11-29T11:11:54.966 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:11:54.981 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:11:54.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-11-29T11:11:55.067 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:11:55.067 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:11:55.283 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 3/3 2019-11-29T11:11:55.395 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-11-29T11:11:55.487 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-11-29T11:11:55.957 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7809.g656c8e8.el7.noarch 3/3 2019-11-29T11:11:55.957 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:55.957 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:11:55.957 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:11:55.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:55.958 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:11:55.958 INFO:teuthology.orchestra.run.smithi069.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-11-29T11:11:55.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:55.958 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:11:56.077 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-fuse 2019-11-29T11:11:56.332 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:11:56.377 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:11:56.377 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:11:56.378 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:11:56.378 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:11:56.378 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:11:57.032 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:11:57.357 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-11-29T11:11:57.366 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:11:57.366 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:57.367 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:11:57.379 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python36 for package: 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:11:57.408 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:11:57.409 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:57.409 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-11-29T11:11:57.409 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:57.411 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:57.412 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:57.413 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-11-29T11:11:57.414 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:11:57.414 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-11-29T11:11:57.419 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-11-29T11:11:57.420 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-11-29T11:11:57.780 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:11:57.904 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-11-29T11:11:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:58.003 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:11:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 566 k 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-11-29T11:11:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+4 Dependent packages) 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 10 M 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 49 M 2019-11-29T11:11:58.006 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:11:58.741 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-11-29T11:11:58.765 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:11:58.766 INFO:teuthology.orchestra.run.smithi069.stdout:Total 13 MB/s | 10 MB 00:00 2019-11-29T11:11:58.779 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:11:58.798 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:11:58.926 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:11:58.927 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:11:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-11-29T11:11:59.560 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 5/5 2019-11-29T11:11:59.693 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/5 2019-11-29T11:11:59.818 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-11-29T11:11:59.926 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-11-29T11:12:00.018 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-11-29T11:12:00.436 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-11-29T11:12:00.436 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:00.436 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:00.437 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:12:00.565 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install libcephfs2 2019-11-29T11:12:00.784 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:00.823 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:00.824 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:00.824 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:00.824 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:01.330 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-11-29T11:12:01.460 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:01.784 INFO:teuthology.orchestra.run.smithi163.stdout:Package 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:01.785 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do 2019-11-29T11:12:01.846 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install libcephfs-devel 2019-11-29T11:12:01.872 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-11-29T11:12:02.075 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:02.119 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:02.119 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:02.120 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:02.734 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-11-29T11:12:02.762 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:03.092 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:12:03.093 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:12:03.093 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:03.093 INFO:teuthology.orchestra.run.smithi163.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7809.g656c8e8.el7 for package: 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:12:03.123 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:12:03.124 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package librados-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:03.249 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-11-29T11:12:03.498 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 5/5 2019-11-29T11:12:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:ceph-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/5 2019-11-29T11:12:03.724 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:03.725 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:12:03.726 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:03.726 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:03.726 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:12:03.726 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:12:03.727 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:03.727 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:12:03.727 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 17 k 2019-11-29T11:12:03.727 INFO:teuthology.orchestra.run.smithi163.stdout:Installing for dependencies: 2019-11-29T11:12:03.728 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 153 k 2019-11-29T11:12:03.728 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:03.728 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:12:03.729 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:03.729 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package (+1 Dependent package) 2019-11-29T11:12:03.729 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:03.729 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 170 k 2019-11-29T11:12:03.730 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 679 k 2019-11-29T11:12:03.730 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:12:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-11-29T11:12:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-11-29T11:12:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-11-29T11:12:04.108 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:12:04.108 INFO:teuthology.orchestra.run.smithi163.stdout:Total 445 kB/s | 170 kB 00:00 2019-11-29T11:12:04.111 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:12:04.123 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:12:04.126 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:12:04.141 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:12:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-11-29T11:12:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-11-29T11:12:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:04.470 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:12:04.629 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install libcephfs2 2019-11-29T11:12:04.905 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:04.951 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:05.023 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:librados-devel-15.0.0-7809.g656c8e8.el7.x86_64 1/2 2019-11-29T11:12:05.293 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 2/2 2019-11-29T11:12:05.407 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 1/2 2019-11-29T11:12:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:05.870 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:librados-devel-15.0.0-7809.g656c8e8.el7.x86_64 2/2 2019-11-29T11:12:05.870 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:05.870 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:12:05.871 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:05.871 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:05.871 INFO:teuthology.orchestra.run.smithi163.stdout:Dependency Installed: 2019-11-29T11:12:05.871 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:05.871 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:05.880 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:12:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:Package 2:libcephfs2-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do 2019-11-29T11:12:05.992 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install librados2 2019-11-29T11:12:05.998 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install libcephfs-devel 2019-11-29T11:12:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:06.187 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:06.189 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:06.189 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:06.189 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:06.189 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:06.215 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:06.255 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:06.256 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:06.257 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:06.257 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:06.257 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:06.762 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:06.902 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:07.081 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:12:07.082 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:12:07.082 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:07.083 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7809.g656c8e8.el7 for package: 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 2019-11-29T11:12:07.111 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:12:07.112 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package librados-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:07.232 INFO:teuthology.orchestra.run.smithi163.stdout:Package 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:07.233 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do 2019-11-29T11:12:07.295 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install librbd1 2019-11-29T11:12:07.480 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:07.532 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:07.577 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:07.578 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:07.578 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:07.578 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:07.579 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:07.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:07.704 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:12:07.705 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 17 k 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 153 k 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:12:07.706 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:07.707 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package (+1 Dependent package) 2019-11-29T11:12:07.707 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:07.707 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 170 k 2019-11-29T11:12:07.707 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 679 k 2019-11-29T11:12:07.707 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:12:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:12:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:Total 464 kB/s | 170 kB 00:00 2019-11-29T11:12:08.074 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:12:08.085 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:12:08.088 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:12:08.088 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:12:08.221 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:08.552 INFO:teuthology.orchestra.run.smithi163.stdout:Package 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:08.552 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do 2019-11-29T11:12:08.613 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install python-ceph 2019-11-29T11:12:08.696 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:librados-devel-15.0.0-7809.g656c8e8.el7.x86_64 1/2 2019-11-29T11:12:08.840 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:08.882 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:08.883 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:08.884 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:08.884 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:08.884 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:08.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 2/2 2019-11-29T11:12:09.122 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:libcephfs-devel-15.0.0-7809.g656c8e8.el7.x86_64 1/2 2019-11-29T11:12:09.540 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:09.868 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:12:09.869 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:12:09.869 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:10.247 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:10.452 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:librados-devel-15.0.0-7809.g656c8e8.el7.x86_64 2/2 2019-11-29T11:12:10.453 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:10.453 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:12:10.453 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:10.453 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:10.453 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:12:10.454 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:10.454 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:10.454 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:12:10.478 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:10.479 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:12:10.479 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout: Size 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:12:10.480 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-compat x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 2.2 k 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 2.2 k 2019-11-29T11:12:10.481 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 0 2019-11-29T11:12:10.482 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:12:10.569 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install librados2 2019-11-29T11:12:10.820 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:10.835 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:12:10.837 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:12:10.843 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:12:10.843 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:12:10.864 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:10.865 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:10.866 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:10.866 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:10.867 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:11.322 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:python-ceph-compat-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:11.506 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:11.838 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:python-ceph-compat-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:11.838 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:11.838 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:12:11.838 INFO:teuthology.orchestra.run.smithi163.stdout: python-ceph-compat.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:11.839 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:11.839 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:12:11.840 INFO:teuthology.orchestra.run.smithi069.stdout:Package 2:librados2-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:11.840 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do 2019-11-29T11:12:11.901 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install librbd1 2019-11-29T11:12:11.965 INFO:teuthology.orchestra.run.smithi163:> sudo yum -y install rbd-fuse 2019-11-29T11:12:12.068 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:12.096 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:12.099 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:12.100 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:12.100 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:12.100 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:12.222 INFO:teuthology.orchestra.run.smithi163.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:12.263 INFO:teuthology.orchestra.run.smithi163.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:12.264 INFO:teuthology.orchestra.run.smithi163.stdout: * base: mirror.facebook.net 2019-11-29T11:12:12.265 INFO:teuthology.orchestra.run.smithi163.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:12.265 INFO:teuthology.orchestra.run.smithi163.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:12.265 INFO:teuthology.orchestra.run.smithi163.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:12.683 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:12.906 INFO:teuthology.orchestra.run.smithi163.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:13.005 INFO:teuthology.orchestra.run.smithi069.stdout:Package 2:librbd1-15.0.0-7809.g656c8e8.el7.x86_64 already installed and latest version 2019-11-29T11:12:13.005 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do 2019-11-29T11:12:13.065 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install python-ceph 2019-11-29T11:12:13.232 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:13.245 INFO:teuthology.orchestra.run.smithi163.stdout:Resolving Dependencies 2019-11-29T11:12:13.246 INFO:teuthology.orchestra.run.smithi163.stdout:--> Running transaction check 2019-11-29T11:12:13.246 INFO:teuthology.orchestra.run.smithi163.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:13.266 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:13.267 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:13.267 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:13.268 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:13.268 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:13.622 INFO:teuthology.orchestra.run.smithi163.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:13.845 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:13.852 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:13.852 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies Resolved 2019-11-29T11:12:13.855 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:13.856 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:13.856 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2019-11-29T11:12:13.856 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:13.857 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2019-11-29T11:12:13.857 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 76 k 2019-11-29T11:12:13.857 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:13.857 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2019-11-29T11:12:13.857 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2019-11-29T11:12:13.858 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2019-11-29T11:12:13.858 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:13.858 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 76 k 2019-11-29T11:12:13.858 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 200 k 2019-11-29T11:12:13.858 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading packages: 2019-11-29T11:12:14.168 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:12:14.168 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:12:14.168 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:14.180 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2019-11-29T11:12:14.197 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2019-11-29T11:12:14.199 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded 2019-11-29T11:12:14.199 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2019-11-29T11:12:14.536 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:14.761 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:14.761 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:12:14.762 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:14.762 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:14.762 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository 2019-11-29T11:12:14.762 INFO:teuthology.orchestra.run.smithi069.stdout: Size 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-compat x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 2.2 k 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:14.763 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T11:12:14.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:14.764 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 2.2 k 2019-11-29T11:12:14.764 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 0 2019-11-29T11:12:14.764 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:12:14.810 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : 2:rbd-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:15.058 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:12:15.058 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:12:15.060 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:12:15.060 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:12:15.217 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : 2:rbd-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:15.218 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:15.218 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2019-11-29T11:12:15.218 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:15.218 INFO:teuthology.orchestra.run.smithi163.stdout: 2019-11-29T11:12:15.218 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2019-11-29T11:12:15.348 DEBUG:teuthology.parallel:result is None 2019-11-29T11:12:15.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:python-ceph-compat-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:16.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:python-ceph-compat-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:16.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:16.079 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:12:16.079 INFO:teuthology.orchestra.run.smithi069.stdout: python-ceph-compat.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:16.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:16.079 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:12:16.190 INFO:teuthology.orchestra.run.smithi069:> sudo yum -y install rbd-fuse 2019-11-29T11:12:16.483 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:12:16.528 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:12:16.531 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:12:16.531 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:12:16.532 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:12:16.532 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:12:17.173 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:12:17.507 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:12:17.508 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:12:17.508 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 will be installed 2019-11-29T11:12:17.882 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:12:18.111 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:18.111 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:15.0.0-7809.g656c8e8.el7 ceph 76 k 2019-11-29T11:12:18.112 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:12:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:12:18.114 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T11:12:18.114 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:18.115 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 76 k 2019-11-29T11:12:18.115 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 200 k 2019-11-29T11:12:18.115 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:12:18.439 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:12:18.458 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:12:18.459 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:12:18.460 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:12:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : 2:rbd-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : 2:rbd-fuse-15.0.0-7809.g656c8e8.el7.x86_64 1/1 2019-11-29T11:12:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:19.441 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:12:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse.x86_64 2:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:12:19.441 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:12:19.558 DEBUG:teuthology.parallel:result is None 2019-11-29T11:12:19.558 INFO:teuthology.orchestra.run.smithi069:> sudo lsb_release -is 2019-11-29T11:12:19.703 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS 2019-11-29T11:12:19.706 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:12:19.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:12:19.875 INFO:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-11-29T11:12:20.015 INFO:teuthology.orchestra.run.smithi069.stdout:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:20.016 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:20.017 INFO:teuthology.task.install:The correct ceph version 15.0.0-7809.g656c8e8 is installed. 2019-11-29T11:12:20.017 INFO:teuthology.orchestra.run.smithi163:> sudo lsb_release -is 2019-11-29T11:12:20.178 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS 2019-11-29T11:12:20.181 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-29T11:12:20.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:12:20.356 INFO:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-11-29T11:12:20.494 INFO:teuthology.orchestra.run.smithi163.stdout:15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:20.494 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7809.g656c8e8.el7 2019-11-29T11:12:20.494 INFO:teuthology.task.install:The correct ceph version 15.0.0-7809.g656c8e8 is installed. 2019-11-29T11:12:20.494 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-11-29T11:12:20.495 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-11-29T11:12:20.618 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-11-29T11:12:20.741 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-11-29T11:12:20.741 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-11-29T11:12:20.858 INFO:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-11-29T11:12:20.969 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-11-29T11:12:21.091 INFO:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-11-29T11:12:21.189 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-11-29T11:12:21.189 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-11-29T11:12:21.312 INFO:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-11-29T11:12:21.416 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-11-29T11:12:21.536 INFO:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-11-29T11:12:21.644 INFO:teuthology.run_tasks:Running task ceph... 2019-11-29T11:12:21.709 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-11-29T11:12:21.709 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 777 /var/log/ceph 2019-11-29T11:12:21.715 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2019-11-29T11:12:21.834 INFO:tasks.ceph:Disabling ceph logrotate... 2019-11-29T11:12:21.835 INFO:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/logrotate.d/ceph 2019-11-29T11:12:21.839 INFO:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2019-11-29T11:12:21.949 INFO:tasks.ceph:Creating extra log directories... 2019-11-29T11:12:21.950 INFO:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-11-29T11:12:21.954 INFO:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-11-29T11:12:22.051 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-11-29T11:12:22.051 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-11-29T11:12:22.051 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.9251', 'owner': 'scheduled_yuvalif@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi069.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDiH4AntPmRDAGRgVfFgbbYgSMcmsR5Nk76bDfDRzJf4zB+Ri4450gZKaE2j2bZeSCv0Wc/rj2kYGwZkoOEzZkF6cRoqogAzweakrGC35FqyWhBpnk1zsgGSdyLmTQBlDNd0EdgmFGBWev0y9DkQbMEAzOfQCkvMcAudZiqU2sRZ0lzIPXlTA0V37WFtW7QYI0F6JTKHvZmHtkRpsQ5otM+gRK+Jic8CMdnxokAs/QbxqWr4sK6aM/wEMMr3O/pv3t36PyJ0pjyQ9OmBh+TKHMJgYcX/zqGk6L4vzd861vV2IDg0eIEfuEyoXhbnxIbvhJKWChZ+XBXjekDxcDiwIrJ', 'smithi163.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1Dq5pttXqbPFyjKdzabTdCoOCSLFlDCctS6la1WFsZMHJXrdPYg5Qn6zAPG1mH7WZA+MABKvLCAcDi1PcfX2ex5XDUycR2T1mjYNkvW8uhL3nhiT+NKE9wJ6XMKV63te3qcjcNVVAnXgVW1ptc35VghTmHn6lQqJYSe6sTD6+TkwgmGdtOpm7gUFDPO9m/zCt2exAZVtBzH1UzRfR4qVYlulV6eay5sZXxE/y63/Y9bEKy45IRZp8/pHmqn9D+NME7B0ZG46TEEX06BaGszsQAoqzwqX6ggvhJyneSUxqMCqc24BOi36t18ILaQMcbNVZz9Ell1gMlmO4O/1/TNS1'}, 'job_id': '4552875', 'tube': 'smithi', 'suite_branch': 'wip-yuval-master', 'priority': 100, 'first_in_suite': False, 'branch': 'wip-yuval-master', 'suite': 'rgw:verify', 'email': 'yuvalif@yahoo.com', 'archive_path': '/home/teuthworker/archive/yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi/4552875', 'description': 'rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64', 'name': 'yuvalif-2019-11-29_09:46:32-rgw:verify-wip-yuval-master-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64', 'os_type': 'centos', 'overrides': {'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64'}, 's3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'filestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd sloppy crc': True, 'osd objectstore': 'filestore'}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64'}}, 'rgw': {'frontend': 'civetweb', 'compression type': 'random', 'storage classes': 'LUKEWARM, FROZEN', 'ec-data-pool': True}, 'admin_socket': {'branch': 'wip-yuval-master'}, 'workunit': {'sha1': '656c8e8049c2c1acd363143c842c2edf1fe09b64', 'branch': 'wip-yuval-master'}}} 2019-11-29T11:12:22.052 INFO:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-11-29T11:12:22.064 INFO:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-11-29T11:12:22.141 INFO:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/run/ceph 2019-11-29T11:12:22.145 INFO:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2019-11-29T11:12:22.341 DEBUG:teuthology.orchestra.remote:smithi069:/scratch_devs is 72B 2019-11-29T11:12:22.422 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-29T11:12:22.422 INFO:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-11-29T11:12:22.492 INFO:teuthology.orchestra.run.smithi069.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 109623 Links: 1 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2019-11-29 11:11:12.618584586 +0000 2019-11-29T11:12:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2019-11-29 11:09:21.571207949 +0000 2019-11-29T11:12:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2019-11-29 11:09:21.571207949 +0000 2019-11-29T11:12:22.494 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2019-11-29T11:12:22.508 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2019-11-29T11:12:22.509 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2019-11-29T11:12:22.509 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes (512 B) copied, 0.000307796 s, 1.7 MB/s 2019-11-29T11:12:22.514 INFO:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-11-29T11:12:22.662 INFO:teuthology.orchestra.run.smithi069.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-11-29T11:12:22.662 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:22.662 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 102142 Links: 1 2019-11-29T11:12:22.662 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:22.662 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:22.663 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2019-11-29 11:11:12.618584586 +0000 2019-11-29T11:12:22.663 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2019-11-29 11:09:22.042197618 +0000 2019-11-29T11:12:22.663 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2019-11-29 11:09:22.042197618 +0000 2019-11-29T11:12:22.663 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2019-11-29T11:12:22.691 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2019-11-29T11:12:22.692 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2019-11-29T11:12:22.692 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes (512 B) copied, 0.000463132 s, 1.1 MB/s 2019-11-29T11:12:22.695 INFO:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-11-29T11:12:22.758 INFO:teuthology.orchestra.run.smithi069.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-11-29T11:12:22.758 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:22.759 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 102187 Links: 1 2019-11-29T11:12:22.759 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:22.759 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:22.759 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2019-11-29 11:11:12.618584586 +0000 2019-11-29T11:12:22.760 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2019-11-29 11:09:22.539186717 +0000 2019-11-29T11:12:22.760 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2019-11-29 11:09:22.539186717 +0000 2019-11-29T11:12:22.760 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2019-11-29T11:12:22.779 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2019-11-29T11:12:22.779 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2019-11-29T11:12:22.780 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes (512 B) copied, 0.000314927 s, 1.6 MB/s 2019-11-29T11:12:22.783 INFO:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-11-29T11:12:22.845 INFO:teuthology.orchestra.run.smithi069.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-11-29T11:12:22.845 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:22.846 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 111631 Links: 1 2019-11-29T11:12:22.846 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:22.846 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:22.846 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2019-11-29 11:11:12.618584586 +0000 2019-11-29T11:12:22.846 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2019-11-29 11:09:23.004176518 +0000 2019-11-29T11:12:22.847 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2019-11-29 11:09:23.004176518 +0000 2019-11-29T11:12:22.847 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2019-11-29T11:12:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2019-11-29T11:12:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2019-11-29T11:12:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes (512 B) copied, 0.000285852 s, 1.8 MB/s 2019-11-29T11:12:22.874 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-11-29T11:12:22.874 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-29T11:12:22.875 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-11-29T11:12:22.875 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:22.982 DEBUG:teuthology.orchestra.remote:smithi163:/scratch_devs is 72B 2019-11-29T11:12:23.070 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-29T11:12:23.071 INFO:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-11-29T11:12:23.157 INFO:teuthology.orchestra.run.smithi163.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-11-29T11:12:23.158 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:23.158 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 102070 Links: 1 2019-11-29T11:12:23.158 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:23.158 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:23.158 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2019-11-29 11:11:10.028007796 +0000 2019-11-29T11:12:23.159 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2019-11-29 11:09:21.550159346 +0000 2019-11-29T11:12:23.159 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2019-11-29 11:09:21.550159346 +0000 2019-11-29T11:12:23.159 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2019-11-29T11:12:23.189 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2019-11-29T11:12:23.189 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2019-11-29T11:12:23.189 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes (512 B) copied, 0.000440319 s, 1.2 MB/s 2019-11-29T11:12:23.193 INFO:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-11-29T11:12:23.346 INFO:teuthology.orchestra.run.smithi163.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-11-29T11:12:23.346 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:23.347 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 96099 Links: 1 2019-11-29T11:12:23.347 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:23.347 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:23.347 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2019-11-29 11:11:10.028007796 +0000 2019-11-29T11:12:23.347 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2019-11-29 11:09:22.039146665 +0000 2019-11-29T11:12:23.348 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2019-11-29 11:09:22.039146665 +0000 2019-11-29T11:12:23.348 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2019-11-29T11:12:23.375 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2019-11-29T11:12:23.375 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2019-11-29T11:12:23.375 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes (512 B) copied, 0.000452356 s, 1.1 MB/s 2019-11-29T11:12:23.381 INFO:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-11-29T11:12:23.460 INFO:teuthology.orchestra.run.smithi163.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-11-29T11:12:23.460 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:23.464 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 102843 Links: 1 2019-11-29T11:12:23.464 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:23.464 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:23.466 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2019-11-29 11:11:10.028007796 +0000 2019-11-29T11:12:23.467 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2019-11-29 11:09:22.506134555 +0000 2019-11-29T11:12:23.467 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2019-11-29 11:09:22.506134555 +0000 2019-11-29T11:12:23.467 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2019-11-29T11:12:23.489 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2019-11-29T11:12:23.489 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2019-11-29T11:12:23.489 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes (512 B) copied, 0.000457852 s, 1.1 MB/s 2019-11-29T11:12:23.494 INFO:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-11-29T11:12:23.572 INFO:teuthology.orchestra.run.smithi163.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-11-29T11:12:23.572 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-29T11:12:23.572 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 102869 Links: 1 2019-11-29T11:12:23.572 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-29T11:12:23.572 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2019-11-29T11:12:23.573 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2019-11-29 11:11:10.028007796 +0000 2019-11-29T11:12:23.573 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2019-11-29 11:09:22.992121953 +0000 2019-11-29T11:12:23.573 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2019-11-29 11:09:22.992121953 +0000 2019-11-29T11:12:23.573 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2019-11-29T11:12:23.598 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2019-11-29T11:12:23.598 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2019-11-29T11:12:23.598 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes (512 B) copied, 0.000473005 s, 1.1 MB/s 2019-11-29T11:12:23.603 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-11-29T11:12:23.603 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-29T11:12:23.603 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-11-29T11:12:23.604 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:23.604 INFO:tasks.ceph:Generating config... 2019-11-29T11:12:23.605 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-11-29T11:12:23.605 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-11-29T11:12:23.605 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-11-29T11:12:23.606 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-11-29T11:12:23.606 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-11-29T11:12:23.606 INFO:tasks.ceph:[osd] debug ms = 1 2019-11-29T11:12:23.606 INFO:tasks.ceph:[osd] debug journal = 20 2019-11-29T11:12:23.606 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-11-29T11:12:23.606 INFO:tasks.ceph:[osd] debug osd = 25 2019-11-29T11:12:23.606 INFO:tasks.ceph:[osd] osd objectstore = filestore 2019-11-29T11:12:23.607 INFO:tasks.ceph:[osd] osd sloppy crc = True 2019-11-29T11:12:23.607 INFO:tasks.ceph:[osd] debug filestore = 20 2019-11-29T11:12:23.607 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-11-29T11:12:23.607 INFO:tasks.ceph:[mon] debug mon = 20 2019-11-29T11:12:23.607 INFO:tasks.ceph:[mon] debug paxos = 20 2019-11-29T11:12:23.607 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-11-29T11:12:23.607 INFO:tasks.ceph:[mon] debug ms = 1 2019-11-29T11:12:23.608 INFO:tasks.ceph:[client] debug rgw = 20 2019-11-29T11:12:23.608 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-11-29T11:12:23.608 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-11-29T11:12:23.608 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-11-29T11:12:23.609 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-11-29T11:12:23.609 INFO:tasks.ceph:Setting up mon.a... 2019-11-29T11:12:23.609 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-11-29T11:12:23.814 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.keyring 2019-11-29T11:12:23.816 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-11-29T11:12:23.972 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-29T11:12:24.090 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.69'), ('mon.c', '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]'), ('mon.b', '172.21.15.163')] 2019-11-29T11:12:24.090 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.69 --addv c '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]' --add b 172.21.15.163 --print /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:24.245 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:24.245 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: generated fsid c38ae7c0-5696-449e-b9f9-b89969aec282 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 0 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:fsid c38ae7c0-5696-449e-b9f9-b89969aec282 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:last_changed 2019-11-29T11:12:24.257175+0000 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:created 2019-11-29T11:12:24.257175+0000 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:min_mon_release 0 (unknown) 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2019-11-29T11:12:24.246 INFO:teuthology.orchestra.run.smithi069.stdout:1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2019-11-29T11:12:24.247 INFO:teuthology.orchestra.run.smithi069.stdout:2: [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] mon.c 2019-11-29T11:12:24.247 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-11-29T11:12:24.247 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c38ae7c0-5696-449e-b9f9-b89969aec282... 2019-11-29T11:12:24.248 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-11-29T11:12:24.260 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-11-29T11:12:24.476 INFO:tasks.ceph:Creating admin key on mon.a... 2019-11-29T11:12:24.476 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-11-29T11:12:24.613 INFO:tasks.ceph:Copying monmap to all nodes... 2019-11-29T11:12:24.682 DEBUG:teuthology.orchestra.remote:smithi069:/etc/ceph/ceph.keyring is 206B 2019-11-29T11:12:24.814 DEBUG:teuthology.orchestra.remote:smithi069:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-11-29T11:12:24.881 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:12:24.881 INFO:teuthology.orchestra.run.smithi069:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-29T11:12:24.974 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:25.025 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:12:25.025 INFO:teuthology.orchestra.run.smithi163:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-29T11:12:25.176 INFO:teuthology.orchestra.run.smithi163:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:25.240 INFO:tasks.ceph:Setting up mon nodes... 2019-11-29T11:12:25.240 INFO:tasks.ceph:Setting up mgr nodes... 2019-11-29T11:12:25.240 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-11-29T11:12:25.403 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-11-29T11:12:25.408 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-11-29T11:12:25.620 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-11-29T11:12:25.624 INFO:tasks.ceph:Setting up mds nodes... 2019-11-29T11:12:25.624 INFO:tasks.ceph_client:Setting up client nodes... 2019-11-29T11:12:25.624 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-11-29T11:12:25.776 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-11-29T11:12:25.811 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-11-29T11:12:25.961 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-11-29T11:12:25.995 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-11-29T11:12:25.995 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-11-29T11:12:25.995 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-11-29T11:12:26.116 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:26.116 INFO:tasks.ceph:{} 2019-11-29T11:12:26.116 INFO:tasks.ceph:osd.0 2019-11-29T11:12:26.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:12:26.125 INFO:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-11-29T11:12:26.699 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:26.700 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:26.701 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:26.701 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:26.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2019-11-29T11:12:26.703 INFO:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-11-29T11:12:26.878 INFO:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-11-29T11:12:26.999 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-11-29T11:12:27.100 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:27.100 INFO:tasks.ceph:{} 2019-11-29T11:12:27.101 INFO:tasks.ceph:osd.1 2019-11-29T11:12:27.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:12:27.101 INFO:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:27.712 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:27.713 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:27.713 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:27.713 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:27.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2019-11-29T11:12:27.714 INFO:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-11-29T11:12:27.872 INFO:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-11-29T11:12:27.994 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-11-29T11:12:28.098 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:28.099 INFO:tasks.ceph:{} 2019-11-29T11:12:28.099 INFO:tasks.ceph:osd.2 2019-11-29T11:12:28.099 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:12:28.099 INFO:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-11-29T11:12:28.701 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:28.701 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:28.701 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:28.701 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:28.702 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:28.702 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:28.702 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:28.702 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:28.703 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:28.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2019-11-29T11:12:28.704 INFO:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-11-29T11:12:28.882 INFO:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-11-29T11:12:28.999 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-11-29T11:12:29.109 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:29.109 INFO:tasks.ceph:{} 2019-11-29T11:12:29.109 INFO:tasks.ceph:osd.3 2019-11-29T11:12:29.110 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:12:29.110 INFO:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-11-29T11:12:29.718 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:29.718 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:29.718 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:29.719 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:29.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2019-11-29T11:12:29.720 INFO:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-11-29T11:12:29.857 INFO:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-11-29T11:12:29.944 INFO:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.119+0000 7f68596b2b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-11-29T11:12:30.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.119+0000 7f68596b2b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-11-29T11:12:30.124 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.141+0000 7f68596b2b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:30.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.141+0000 7f68596b2b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:30.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.392+0000 7f68596b2b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:30.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.395+0000 7f68596b2b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:30.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.395+0000 7f68596b2b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:30.404 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-11-29T11:12:30.516 INFO:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:30.681 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.694+0000 7fcb98001b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-11-29T11:12:30.681 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.698+0000 7fcb98001b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-11-29T11:12:30.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.732+0000 7fcb98001b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:30.715 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.732+0000 7fcb98001b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:30.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.983+0000 7fcb98001b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:30.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.984+0000 7fcb98001b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:30.967 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:30.984+0000 7fcb98001b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:30.985 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-11-29T11:12:31.099 INFO:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:31.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.245+0000 7f7502024b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-11-29T11:12:31.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.245+0000 7f7502024b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-11-29T11:12:31.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.259+0000 7f7502024b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:31.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.259+0000 7f7502024b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:31.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.394+0000 7f7502024b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:31.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.395+0000 7f7502024b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:31.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.395+0000 7f7502024b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:31.392 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-11-29T11:12:31.495 INFO:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:31.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.721+0000 7f8515fb5b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-11-29T11:12:31.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.721+0000 7f8515fb5b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-11-29T11:12:31.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.736+0000 7f8515fb5b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:31.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.736+0000 7f8515fb5b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:31.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.883+0000 7f8515fb5b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:31.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.883+0000 7f8515fb5b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:31.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:12:31.883+0000 7f8515fb5b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:31.882 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-11-29T11:12:31.985 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-11-29T11:12:32.104 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:32.104 INFO:tasks.ceph:{} 2019-11-29T11:12:32.104 INFO:tasks.ceph:osd.4 2019-11-29T11:12:32.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:12:32.105 INFO:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-11-29T11:12:32.717 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:32.717 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:32.717 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:32.717 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:32.718 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:32.718 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:32.718 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:32.718 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:32.718 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:32.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2019-11-29T11:12:32.720 INFO:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-11-29T11:12:32.900 INFO:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-11-29T11:12:32.989 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-11-29T11:12:33.065 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:33.065 INFO:tasks.ceph:{} 2019-11-29T11:12:33.065 INFO:tasks.ceph:osd.5 2019-11-29T11:12:33.065 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:12:33.065 INFO:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-11-29T11:12:33.625 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:33.625 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:33.625 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:33.626 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:33.628 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2019-11-29T11:12:33.628 INFO:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-11-29T11:12:33.787 INFO:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-11-29T11:12:33.893 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-11-29T11:12:33.976 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:33.977 INFO:tasks.ceph:{} 2019-11-29T11:12:33.977 INFO:tasks.ceph:osd.6 2019-11-29T11:12:33.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:12:33.977 INFO:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-11-29T11:12:34.568 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:34.569 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:34.569 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:34.569 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:34.569 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:34.570 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:34.570 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:34.570 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:34.570 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:34.571 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2019-11-29T11:12:34.572 INFO:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-11-29T11:12:34.716 INFO:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-11-29T11:12:34.818 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-11-29T11:12:34.925 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-29T11:12:34.926 INFO:tasks.ceph:{} 2019-11-29T11:12:34.926 INFO:tasks.ceph:osd.7 2019-11-29T11:12:34.926 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:12:34.926 INFO:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-11-29T11:12:35.504 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-11-29T11:12:35.505 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=0, sparse=0 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-29T11:12:35.506 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-29T11:12:35.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2019-11-29T11:12:35.508 INFO:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-11-29T11:12:35.643 INFO:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-11-29T11:12:35.744 INFO:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:35.879 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:35.881+0000 7f3e4b9d0b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-11-29T11:12:35.879 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:35.881+0000 7f3e4b9d0b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-11-29T11:12:35.899 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:35.902+0000 7f3e4b9d0b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:35.900 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:35.902+0000 7f3e4b9d0b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:36.130 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.133+0000 7f3e4b9d0b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.135+0000 7f3e4b9d0b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.135+0000 7f3e4b9d0b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:36.154 INFO:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-11-29T11:12:36.264 INFO:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.406+0000 7f607f495b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-11-29T11:12:36.407 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.410+0000 7f607f495b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-11-29T11:12:36.442 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.443+0000 7f607f495b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:36.443 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.443+0000 7f607f495b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:36.690 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.692+0000 7f607f495b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:36.692 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.694+0000 7f607f495b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:36.692 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.694+0000 7f607f495b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:36.719 INFO:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-11-29T11:12:36.829 INFO:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:36.985 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.987+0000 7f59ca580b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-11-29T11:12:36.986 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:36.988+0000 7f59ca580b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-11-29T11:12:37.006 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.004+0000 7f59ca580b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:37.006 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.004+0000 7f59ca580b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:37.148 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.149+0000 7f59ca580b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:37.148 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.149+0000 7f59ca580b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:37.148 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.149+0000 7f59ca580b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:37.163 INFO:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-11-29T11:12:37.265 INFO:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:37.378 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.381+0000 7ff8d8689b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-11-29T11:12:37.379 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.381+0000 7ff8d8689b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-11-29T11:12:37.389 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.391+0000 7ff8d8689b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:37.389 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.391+0000 7ff8d8689b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-29T11:12:37.518 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.520+0000 7ff8d8689b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:12:37.518 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.520+0000 7ff8d8689b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:37.518 INFO:teuthology.orchestra.run.smithi163.stderr:2019-11-29T11:12:37.520+0000 7ff8d8689b40 -1 journal do_read_entry(4096): bad header magic 2019-11-29T11:12:37.532 INFO:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-11-29T11:12:37.612 INFO:tasks.ceph:Reading keys from all nodes... 2019-11-29T11:12:37.612 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:12:37.701 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.IoDKvmEPA2 2019-11-29T11:12:37.702 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.IoDKvmEPA2 2019-11-29T11:12:37.806 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.IoDKvmEPA2 2019-11-29T11:12:37.973 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.IoDKvmEPA2 is 56B 2019-11-29T11:12:38.052 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.IoDKvmEPA2 2019-11-29T11:12:38.118 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:12:38.275 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.VjkMG46YXq 2019-11-29T11:12:38.275 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.VjkMG46YXq 2019-11-29T11:12:38.389 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.VjkMG46YXq 2019-11-29T11:12:38.588 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.VjkMG46YXq is 56B 2019-11-29T11:12:38.665 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.VjkMG46YXq 2019-11-29T11:12:38.725 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:12:38.875 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.gf8iCV6HGq 2019-11-29T11:12:38.875 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.gf8iCV6HGq 2019-11-29T11:12:38.994 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.gf8iCV6HGq 2019-11-29T11:12:39.196 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.gf8iCV6HGq is 56B 2019-11-29T11:12:39.298 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.gf8iCV6HGq 2019-11-29T11:12:39.379 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:12:39.467 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.Ppd0ChmZWp 2019-11-29T11:12:39.468 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.Ppd0ChmZWp 2019-11-29T11:12:39.571 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.Ppd0ChmZWp 2019-11-29T11:12:39.742 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.Ppd0ChmZWp is 56B 2019-11-29T11:12:39.821 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.Ppd0ChmZWp 2019-11-29T11:12:39.896 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:12:39.966 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.6DITi6A1Ip 2019-11-29T11:12:39.966 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.6DITi6A1Ip 2019-11-29T11:12:40.057 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.6DITi6A1Ip 2019-11-29T11:12:40.190 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.6DITi6A1Ip is 56B 2019-11-29T11:12:40.247 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.6DITi6A1Ip 2019-11-29T11:12:40.300 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:12:40.393 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.PvfZ9BzrRg 2019-11-29T11:12:40.394 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.PvfZ9BzrRg 2019-11-29T11:12:40.493 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.PvfZ9BzrRg 2019-11-29T11:12:40.655 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.PvfZ9BzrRg is 56B 2019-11-29T11:12:40.751 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.PvfZ9BzrRg 2019-11-29T11:12:40.826 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:12:41.001 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.6IifaQi759 2019-11-29T11:12:41.002 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.6IifaQi759 2019-11-29T11:12:41.115 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.6IifaQi759 2019-11-29T11:12:41.323 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.6IifaQi759 is 56B 2019-11-29T11:12:41.444 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.6IifaQi759 2019-11-29T11:12:41.532 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:12:41.606 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.d3aO7CrbMF 2019-11-29T11:12:41.607 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.d3aO7CrbMF 2019-11-29T11:12:41.697 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.d3aO7CrbMF 2019-11-29T11:12:41.855 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.d3aO7CrbMF is 56B 2019-11-29T11:12:41.919 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.d3aO7CrbMF 2019-11-29T11:12:41.970 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:12:42.111 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.sVSEPVAr6Z 2019-11-29T11:12:42.111 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.sVSEPVAr6Z 2019-11-29T11:12:42.198 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.sVSEPVAr6Z 2019-11-29T11:12:42.345 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.sVSEPVAr6Z is 56B 2019-11-29T11:12:42.403 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.sVSEPVAr6Z 2019-11-29T11:12:42.453 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:12:42.601 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.hvEKB667fE 2019-11-29T11:12:42.601 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.hvEKB667fE 2019-11-29T11:12:42.698 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.hvEKB667fE 2019-11-29T11:12:42.868 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.hvEKB667fE is 56B 2019-11-29T11:12:42.963 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.hvEKB667fE 2019-11-29T11:12:43.149 DEBUG:teuthology.orchestra.remote:smithi069:/etc/ceph/ceph.client.0.keyring is 59B 2019-11-29T11:12:43.350 DEBUG:teuthology.orchestra.remote:smithi163:/etc/ceph/ceph.client.1.keyring is 59B 2019-11-29T11:12:43.441 INFO:tasks.ceph:Adding keys to all mons... 2019-11-29T11:12:43.441 INFO:teuthology.orchestra.run.smithi069:> sudo tee -a /etc/ceph/ceph.keyring 2019-11-29T11:12:43.447 INFO:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2019-11-29T11:12:43.553 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAZ/eBddMIOGRAATTQbbgRM5eJsbM9sN5AIDg== 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAe/eBdjSciBxAANzxj0U1E9NihNb+IIOm06A== 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAe/eBdhBBuKRAA1rDkJ8q0AQ/Gny9r5oUb9w== 2019-11-29T11:12:43.554 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAf/eBdTUimDhAAxH1IgftyQ9GAxYqo1FV+uA== 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAf/eBdL2EVKxAAon0lNrQb4Xw0jw9wFk+84A== 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAZ/eBdnVgQJRAAz8yXKPiu9EbytSkZ9Qj0gA== 2019-11-29T11:12:43.555 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAj/eBdPYOTNBAAh5JcWVFvtm7uO3CeACzKSQ== 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAk/eBdq1pNGBAAqtjiUWWE0lYALlVRt9S+mA== 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAk/eBdGMfnOhAAwLmz7SWMZorj5UZ/4dCJ8Q== 2019-11-29T11:12:43.556 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2019-11-29T11:12:43.557 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAl/eBdZffCFhAAjPABwJ2kKSHw6s41UjonMw== 2019-11-29T11:12:43.557 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2019-11-29T11:12:43.557 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAZ/eBd1wtVLxAAPC74Cgf1jEXccSwTzkVJdA== 2019-11-29T11:12:43.557 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2019-11-29T11:12:43.557 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAZ/eBd81V+ORAAdKQuKBsOtpOftwsLRE45HA== 2019-11-29T11:12:43.563 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.y] 2019-11-29T11:12:43.563 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAZ/eBddMIOGRAATTQbbgRM5eJsbM9sN5AIDg== 2019-11-29T11:12:43.563 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.0] 2019-11-29T11:12:43.563 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAe/eBdjSciBxAANzxj0U1E9NihNb+IIOm06A== 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.1] 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAe/eBdhBBuKRAA1rDkJ8q0AQ/Gny9r5oUb9w== 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.2] 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAf/eBdTUimDhAAxH1IgftyQ9GAxYqo1FV+uA== 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.3] 2019-11-29T11:12:43.564 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAf/eBdL2EVKxAAon0lNrQb4Xw0jw9wFk+84A== 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.x] 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAZ/eBdnVgQJRAAz8yXKPiu9EbytSkZ9Qj0gA== 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.4] 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAj/eBdPYOTNBAAh5JcWVFvtm7uO3CeACzKSQ== 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.5] 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAk/eBdq1pNGBAAqtjiUWWE0lYALlVRt9S+mA== 2019-11-29T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.6] 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAk/eBdGMfnOhAAwLmz7SWMZorj5UZ/4dCJ8Q== 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.7] 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAl/eBdZffCFhAAjPABwJ2kKSHw6s41UjonMw== 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAZ/eBd1wtVLxAAPC74Cgf1jEXccSwTzkVJdA== 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout:[client.1] 2019-11-29T11:12:43.566 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAZ/eBd81V+ORAAdKQuKBsOtpOftwsLRE45HA== 2019-11-29T11:12:43.567 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-29T11:12:43.572 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-29T11:12:43.755 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:43.760 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:43.929 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:43.946 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.131 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.152 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.340 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.349 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.519 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-29T11:12:44.524 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-29T11:12:44.692 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.697 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.868 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:44.873 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:45.060 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:45.065 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:45.234 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:45.239 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-29T11:12:45.401 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-29T11:12:45.406 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-29T11:12:45.577 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-29T11:12:45.581 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-29T11:12:45.748 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-11-29T11:12:45.748 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-11-29T11:12:45.888 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-29T11:12:46.044 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-11-29T11:12:46.157 INFO:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-11-29T11:12:46.265 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-29T11:12:46.414 INFO:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-11-29T11:12:46.505 INFO:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-11-29T11:12:46.626 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-29T11:12:46.800 INFO:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-11-29T11:12:46.929 INFO:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:46.934 INFO:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-11-29T11:12:47.021 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-11-29T11:12:47.021 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-29T11:12:47.022 INFO:tasks.ceph.mon.a:Restarting daemon 2019-11-29T11:12:47.022 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-11-29T11:12:47.026 INFO:tasks.ceph.mon.a:Started 2019-11-29T11:12:47.027 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-29T11:12:47.027 INFO:tasks.ceph.mon.c:Restarting daemon 2019-11-29T11:12:47.027 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-11-29T11:12:47.031 INFO:tasks.ceph.mon.c:Started 2019-11-29T11:12:47.031 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-29T11:12:47.031 INFO:tasks.ceph.mon.b:Restarting daemon 2019-11-29T11:12:47.032 INFO:teuthology.orchestra.run.smithi163:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-11-29T11:12:47.036 INFO:tasks.ceph.mon.b:Started 2019-11-29T11:12:47.036 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-11-29T11:12:47.037 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-11-29T11:12:47.037 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-11-29T11:12:47.041 INFO:tasks.ceph.mgr.y:Started 2019-11-29T11:12:47.042 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-11-29T11:12:47.042 INFO:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-11-29T11:12:47.046 INFO:tasks.ceph.mgr.x:Started 2019-11-29T11:12:47.046 INFO:tasks.ceph:Setting crush tunables to default 2019-11-29T11:12:47.046 INFO:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd crush tunables default 2019-11-29T11:12:51.032 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:00:03.813 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:12:51.096 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:00:03.869 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:12:51.158 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:00:03.930 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:03.427 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2019-11-29T11:13:03.440 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-11-29T11:13:03.441 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:13:03.493 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.PtxM1OZlUQ 2019-11-29T11:13:03.494 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.PtxM1OZlUQ 2019-11-29T11:13:03.550 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.PtxM1OZlUQ 2019-11-29T11:13:03.694 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.PtxM1OZlUQ is 37B 2019-11-29T11:13:03.753 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.PtxM1OZlUQ 2019-11-29T11:13:03.798 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:13:03.938 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.mI2jrPr0zx 2019-11-29T11:13:03.938 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.mI2jrPr0zx 2019-11-29T11:13:04.015 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.mI2jrPr0zx 2019-11-29T11:13:04.137 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.mI2jrPr0zx is 37B 2019-11-29T11:13:04.186 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.mI2jrPr0zx 2019-11-29T11:13:04.229 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:13:04.353 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.m5yRHCMBAg 2019-11-29T11:13:04.353 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.m5yRHCMBAg 2019-11-29T11:13:04.424 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.m5yRHCMBAg 2019-11-29T11:13:04.539 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.m5yRHCMBAg is 37B 2019-11-29T11:13:04.586 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.m5yRHCMBAg 2019-11-29T11:13:04.629 INFO:teuthology.orchestra.run.smithi069:> mktemp 2019-11-29T11:13:04.745 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.QcAYYvTuk6 2019-11-29T11:13:04.746 INFO:teuthology.orchestra.run.smithi069:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.QcAYYvTuk6 2019-11-29T11:13:04.809 INFO:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.QcAYYvTuk6 2019-11-29T11:13:04.914 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.QcAYYvTuk6 is 37B 2019-11-29T11:13:04.963 INFO:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.QcAYYvTuk6 2019-11-29T11:13:05.011 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:13:05.097 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.nRvGWp3wUC 2019-11-29T11:13:05.098 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.nRvGWp3wUC 2019-11-29T11:13:05.208 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.nRvGWp3wUC 2019-11-29T11:13:05.395 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.nRvGWp3wUC is 37B 2019-11-29T11:13:05.470 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.nRvGWp3wUC 2019-11-29T11:13:05.536 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:13:05.669 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.LBosZGZuff 2019-11-29T11:13:05.669 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.LBosZGZuff 2019-11-29T11:13:05.749 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.LBosZGZuff 2019-11-29T11:13:05.893 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.LBosZGZuff is 37B 2019-11-29T11:13:05.947 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.LBosZGZuff 2019-11-29T11:13:06.012 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:13:06.164 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.ZMX5hU4pKP 2019-11-29T11:13:06.165 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.ZMX5hU4pKP 2019-11-29T11:13:06.257 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.ZMX5hU4pKP 2019-11-29T11:13:06.406 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.ZMX5hU4pKP is 37B 2019-11-29T11:13:06.465 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.ZMX5hU4pKP 2019-11-29T11:13:06.516 INFO:teuthology.orchestra.run.smithi163:> mktemp 2019-11-29T11:13:06.639 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.7pA6cOVCCY 2019-11-29T11:13:06.639 INFO:teuthology.orchestra.run.smithi163:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.7pA6cOVCCY 2019-11-29T11:13:06.720 INFO:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.7pA6cOVCCY 2019-11-29T11:13:06.848 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.7pA6cOVCCY is 37B 2019-11-29T11:13:06.904 INFO:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.7pA6cOVCCY 2019-11-29T11:13:06.961 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 7abe06a2-119e-4520-96ef-de89ca2fcee1 0 2019-11-29T11:13:08.177 INFO:teuthology.orchestra.run.smithi163.stdout:0 2019-11-29T11:13:08.198 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 5850c501-9be4-4248-9e04-40790c9663ef 1 2019-11-29T11:13:09.237 INFO:teuthology.orchestra.run.smithi163.stdout:1 2019-11-29T11:13:09.259 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 1c233289-0e08-4505-9ecf-7aa1ffd4f544 2 2019-11-29T11:13:10.340 INFO:teuthology.orchestra.run.smithi163.stdout:2 2019-11-29T11:13:10.367 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new cb5684dc-55be-4c85-973f-48ee3fa1e673 3 2019-11-29T11:13:11.574 INFO:teuthology.orchestra.run.smithi163.stdout:3 2019-11-29T11:13:11.594 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new fbb2daaa-d3da-4ebd-a6b2-d447e50b2122 4 2019-11-29T11:13:12.691 INFO:teuthology.orchestra.run.smithi163.stdout:4 2019-11-29T11:13:12.712 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 01c813d9-4c09-4c2c-a633-7b9065614dff 5 2019-11-29T11:13:14.133 INFO:teuthology.orchestra.run.smithi163.stdout:5 2019-11-29T11:13:14.159 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 8401363e-0bb4-45ce-9f14-1d2c58ffec6d 6 2019-11-29T11:13:15.267 INFO:teuthology.orchestra.run.smithi163.stdout:6 2019-11-29T11:13:15.291 INFO:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new f5cfa113-2089-4f74-8db4-cdb1ab325122 7 2019-11-29T11:13:16.534 INFO:teuthology.orchestra.run.smithi163.stdout:7 2019-11-29T11:13:16.560 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.561 INFO:tasks.ceph.osd.0:Restarting daemon 2019-11-29T11:13:16.561 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-11-29T11:13:16.565 INFO:tasks.ceph.osd.0:Started 2019-11-29T11:13:16.566 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.566 INFO:tasks.ceph.osd.1:Restarting daemon 2019-11-29T11:13:16.566 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-11-29T11:13:16.570 INFO:tasks.ceph.osd.1:Started 2019-11-29T11:13:16.570 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.570 INFO:tasks.ceph.osd.2:Restarting daemon 2019-11-29T11:13:16.570 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-11-29T11:13:16.574 INFO:tasks.ceph.osd.2:Started 2019-11-29T11:13:16.574 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.574 INFO:tasks.ceph.osd.3:Restarting daemon 2019-11-29T11:13:16.574 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-11-29T11:13:16.579 INFO:tasks.ceph.osd.3:Started 2019-11-29T11:13:16.579 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.579 INFO:tasks.ceph.osd.4:Restarting daemon 2019-11-29T11:13:16.579 INFO:teuthology.orchestra.run.smithi163:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-11-29T11:13:16.584 INFO:tasks.ceph.osd.4:Started 2019-11-29T11:13:16.584 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.584 INFO:tasks.ceph.osd.5:Restarting daemon 2019-11-29T11:13:16.585 INFO:teuthology.orchestra.run.smithi163:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-11-29T11:13:16.590 INFO:tasks.ceph.osd.5:Started 2019-11-29T11:13:16.590 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.591 INFO:tasks.ceph.osd.6:Restarting daemon 2019-11-29T11:13:16.591 INFO:teuthology.orchestra.run.smithi163:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-11-29T11:13:16.595 INFO:tasks.ceph.osd.6:Started 2019-11-29T11:13:16.596 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-29T11:13:16.596 INFO:tasks.ceph.osd.7:Restarting daemon 2019-11-29T11:13:16.596 INFO:teuthology.orchestra.run.smithi163:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-11-29T11:13:16.600 INFO:tasks.ceph.osd.7:Started 2019-11-29T11:13:16.600 INFO:tasks.ceph:Waiting for OSDs to come up 2019-11-29T11:13:17.407 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:18.446 INFO:teuthology.misc.health.smithi069.stdout: 2019-11-29T11:13:18.446 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":10,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:16.373912+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:18.466 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-11-29T11:13:20.337 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:20.330+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:20.373 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:20.364+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:20.410 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:20.402+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:20.436 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:20.427+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:20.444 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:20.452+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:20.730 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:20.736+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:21.035 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:21.047+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:21.114 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:21.125+0000 9b10800 -1 Falling back to public interface 2019-11-29T11:13:21.284 INFO:tasks.ceph.osd.4.smithi163.stderr:==00:00:00:04.556 9734== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.288 INFO:tasks.ceph.osd.5.smithi163.stderr:==00:00:00:04.564 9733== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.291 INFO:tasks.ceph.osd.7.smithi163.stderr:==00:00:00:04.562 9735== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.339 INFO:tasks.ceph.osd.1.smithi069.stderr:==00:00:00:04.683 9723== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.349 INFO:tasks.ceph.osd.6.smithi163.stderr:==00:00:00:04.610 9736== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.634 INFO:tasks.ceph.osd.0.smithi069.stderr:==00:00:00:04.951 9729== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:21.669 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:21.650+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:21.670 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:21.647+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:21.682 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:21.656+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:21.685 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:21.667+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.689 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:21.664+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.695 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:21.667+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.698 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:21.670+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:21.671+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:21.674+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.722 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:21.724+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:21.745 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:21.739+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.749 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:21.742+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:21.714+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:21.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:21.729+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:21.733+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:21.926 INFO:tasks.ceph.osd.2.smithi069.stderr:==00:00:00:05.249 9728== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:22.013 INFO:tasks.ceph.osd.3.smithi069.stderr:==00:00:00:05.319 9730== Warning: unimplemented fcntl command: 1036 2019-11-29T11:13:22.118 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:22.124+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:22.142 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:22.141+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:22.144+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:22.198+0000 9b10800 -1 osd.4 0 log_to_monitors {default=true} 2019-11-29T11:13:22.254 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:22.254+0000 9b10800 -1 osd.5 0 log_to_monitors {default=true} 2019-11-29T11:13:22.268 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:22.267+0000 9b10800 -1 osd.7 0 log_to_monitors {default=true} 2019-11-29T11:13:22.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:22.282+0000 9b10800 -1 osd.6 0 log_to_monitors {default=true} 2019-11-29T11:13:22.327 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:22.333+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:22.347 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:22.366+0000 9b10800 -1 osd.1 0 log_to_monitors {default=true} 2019-11-29T11:13:22.353 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:22.351+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.370 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:22.354+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.483 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:22.483+0000 9b10800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-29T11:13:22.486 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:22.507+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:22.512+0000 9b10800 -1 journal do_read_entry(8192): bad header magic 2019-11-29T11:13:22.742 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:22.760+0000 9b10800 -1 osd.0 0 log_to_monitors {default=true} 2019-11-29T11:13:22.887 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:22.906+0000 9b10800 -1 osd.2 0 log_to_monitors {default=true} 2019-11-29T11:13:23.129 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:23.142+0000 9b10800 -1 osd.3 0 log_to_monitors {default=true} 2019-11-29T11:13:24.948 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T11:13:24.964+0000 29a5c700 -1 osd.1 0 waiting for initial osdmap 2019-11-29T11:13:24.956 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T11:13:24.971+0000 29a5c700 -1 osd.0 0 waiting for initial osdmap 2019-11-29T11:13:24.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T11:13:24.974+0000 29a5c700 -1 osd.6 0 waiting for initial osdmap 2019-11-29T11:13:24.980 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:13:24.997+0000 29a5c700 -1 osd.2 0 waiting for initial osdmap 2019-11-29T11:13:24.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T11:13:24.978+0000 29a5c700 -1 osd.5 0 waiting for initial osdmap 2019-11-29T11:13:24.992 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T11:13:24.990+0000 2965c700 -1 osd.4 0 waiting for initial osdmap 2019-11-29T11:13:25.033 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T11:13:25.022+0000 29a5c700 -1 osd.7 0 waiting for initial osdmap 2019-11-29T11:13:25.278 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:26.168 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T11:13:26.185+0000 29a5c700 -1 osd.3 0 waiting for initial osdmap 2019-11-29T11:13:26.180 INFO:teuthology.misc.health.smithi069.stdout: 2019-11-29T11:13:26.180 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":13,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:25.728919+0000","last_up_change":"2019-11-29T11:13:25.728919+0000","last_in_change":"2019-11-29T11:13:25.728919+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":9729},{"type":"v1","addr":"172.21.15.69:6811","nonce":9729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":9729},{"type":"v1","addr":"172.21.15.69:6813","nonce":9729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":9729},{"type":"v1","addr":"172.21.15.69:6817","nonce":9729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":9729},{"type":"v1","addr":"172.21.15.69:6815","nonce":9729}]},"public_addr":"172.21.15.69:6811/9729","cluster_addr":"172.21.15.69:6813/9729","heartbeat_back_addr":"172.21.15.69:6817/9729","heartbeat_front_addr":"172.21.15.69:6815/9729","state":["exists","up"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":9723},{"type":"v1","addr":"172.21.15.69:6803","nonce":9723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":9723},{"type":"v1","addr":"172.21.15.69:6805","nonce":9723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":9723},{"type":"v1","addr":"172.21.15.69:6809","nonce":9723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":9723},{"type":"v1","addr":"172.21.15.69:6807","nonce":9723}]},"public_addr":"172.21.15.69:6803/9723","cluster_addr":"172.21.15.69:6805/9723","heartbeat_back_addr":"172.21.15.69:6809/9723","heartbeat_front_addr":"172.21.15.69:6807/9723","state":["exists","up"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":9728},{"type":"v1","addr":"172.21.15.69:6819","nonce":9728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":9728},{"type":"v1","addr":"172.21.15.69:6821","nonce":9728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":9728},{"type":"v1","addr":"172.21.15.69:6825","nonce":9728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":9728},{"type":"v1","addr":"172.21.15.69:6823","nonce":9728}]},"public_addr":"172.21.15.69:6819/9728","cluster_addr":"172.21.15.69:6821/9728","heartbeat_back_addr":"172.21.15.69:6825/9728","heartbeat_front_addr":"172.21.15.69:6823/9728","state":["exists","up"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":9734},{"type":"v1","addr":"172.21.15.163:6801","nonce":9734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":9734},{"type":"v1","addr":"172.21.15.163:6803","nonce":9734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":9734},{"type":"v1","addr":"172.21.15.163:6807","nonce":9734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":9734},{"type":"v1","addr":"172.21.15.163:6805","nonce":9734}]},"public_addr":"172.21.15.163:6801/9734","cluster_addr":"172.21.15.163:6803/9734","heartbeat_back_addr":"172.21.15.163:6807/9734","heartbeat_front_addr":"172.21.15.163:6805/9734","state":["exists","up"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":9733},{"type":"v1","addr":"172.21.15.163:6809","nonce":9733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":9733},{"type":"v1","addr":"172.21.15.163:6811","nonce":9733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":9733},{"type":"v1","addr":"172.21.15.163:6815","nonce":9733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":9733},{"type":"v1","addr":"172.21.15.163:6813","nonce":9733}]},"public_addr":"172.21.15.163:6809/9733","cluster_addr":"172.21.15.163:6811/9733","heartbeat_back_addr":"172.21.15.163:6815/9733","heartbeat_front_addr":"172.21.15.163:6813/9733","state":["exists","up"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":9736},{"type":"v1","addr":"172.21.15.163:6825","nonce":9736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":9736},{"type":"v1","addr":"172.21.15.163:6827","nonce":9736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":9736},{"type":"v1","addr":"172.21.15.163:6831","nonce":9736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":9736},{"type":"v1","addr":"172.21.15.163:6829","nonce":9736}]},"public_addr":"172.21.15.163:6825/9736","cluster_addr":"172.21.15.163:6827/9736","heartbeat_back_addr":"172.21.15.163:6831/9736","heartbeat_front_addr":"172.21.15.163:6829/9736","state":["exists","up"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":9735},{"type":"v1","addr":"172.21.15.163:6817","nonce":9735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":9735},{"type":"v1","addr":"172.21.15.163:6819","nonce":9735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":9735},{"type":"v1","addr":"172.21.15.163:6824","nonce":9735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":9735},{"type":"v1","addr":"172.21.15.163:6821","nonce":9735}]},"public_addr":"172.21.15.163:6817/9735","cluster_addr":"172.21.15.163:6819/9735","heartbeat_back_addr":"172.21.15.163:6824/9735","heartbeat_front_addr":"172.21.15.163:6821/9735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:26.202 DEBUG:teuthology.misc:7 of 8 OSDs are up 2019-11-29T11:13:33.030 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:33.954 INFO:teuthology.misc.health.smithi069.stdout: 2019-11-29T11:13:33.954 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":15,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:28.303240+0000","last_up_change":"2019-11-29T11:13:27.035374+0000","last_in_change":"2019-11-29T11:13:27.035374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":9729},{"type":"v1","addr":"172.21.15.69:6811","nonce":9729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":9729},{"type":"v1","addr":"172.21.15.69:6813","nonce":9729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":9729},{"type":"v1","addr":"172.21.15.69:6817","nonce":9729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":9729},{"type":"v1","addr":"172.21.15.69:6815","nonce":9729}]},"public_addr":"172.21.15.69:6811/9729","cluster_addr":"172.21.15.69:6813/9729","heartbeat_back_addr":"172.21.15.69:6817/9729","heartbeat_front_addr":"172.21.15.69:6815/9729","state":["exists","up"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":9723},{"type":"v1","addr":"172.21.15.69:6803","nonce":9723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":9723},{"type":"v1","addr":"172.21.15.69:6805","nonce":9723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":9723},{"type":"v1","addr":"172.21.15.69:6809","nonce":9723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":9723},{"type":"v1","addr":"172.21.15.69:6807","nonce":9723}]},"public_addr":"172.21.15.69:6803/9723","cluster_addr":"172.21.15.69:6805/9723","heartbeat_back_addr":"172.21.15.69:6809/9723","heartbeat_front_addr":"172.21.15.69:6807/9723","state":["exists","up"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":9728},{"type":"v1","addr":"172.21.15.69:6819","nonce":9728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":9728},{"type":"v1","addr":"172.21.15.69:6821","nonce":9728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":9728},{"type":"v1","addr":"172.21.15.69:6825","nonce":9728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":9728},{"type":"v1","addr":"172.21.15.69:6823","nonce":9728}]},"public_addr":"172.21.15.69:6819/9728","cluster_addr":"172.21.15.69:6821/9728","heartbeat_back_addr":"172.21.15.69:6825/9728","heartbeat_front_addr":"172.21.15.69:6823/9728","state":["exists","up"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":9730},{"type":"v1","addr":"172.21.15.69:6827","nonce":9730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":9730},{"type":"v1","addr":"172.21.15.69:6829","nonce":9730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":9730},{"type":"v1","addr":"172.21.15.69:6833","nonce":9730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":9730},{"type":"v1","addr":"172.21.15.69:6831","nonce":9730}]},"public_addr":"172.21.15.69:6827/9730","cluster_addr":"172.21.15.69:6829/9730","heartbeat_back_addr":"172.21.15.69:6833/9730","heartbeat_front_addr":"172.21.15.69:6831/9730","state":["exists","up"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":9734},{"type":"v1","addr":"172.21.15.163:6801","nonce":9734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":9734},{"type":"v1","addr":"172.21.15.163:6803","nonce":9734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":9734},{"type":"v1","addr":"172.21.15.163:6807","nonce":9734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":9734},{"type":"v1","addr":"172.21.15.163:6805","nonce":9734}]},"public_addr":"172.21.15.163:6801/9734","cluster_addr":"172.21.15.163:6803/9734","heartbeat_back_addr":"172.21.15.163:6807/9734","heartbeat_front_addr":"172.21.15.163:6805/9734","state":["exists","up"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":9733},{"type":"v1","addr":"172.21.15.163:6809","nonce":9733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":9733},{"type":"v1","addr":"172.21.15.163:6811","nonce":9733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":9733},{"type":"v1","addr":"172.21.15.163:6815","nonce":9733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":9733},{"type":"v1","addr":"172.21.15.163:6813","nonce":9733}]},"public_addr":"172.21.15.163:6809/9733","cluster_addr":"172.21.15.163:6811/9733","heartbeat_back_addr":"172.21.15.163:6815/9733","heartbeat_front_addr":"172.21.15.163:6813/9733","state":["exists","up"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":9736},{"type":"v1","addr":"172.21.15.163:6825","nonce":9736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":9736},{"type":"v1","addr":"172.21.15.163:6827","nonce":9736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":9736},{"type":"v1","addr":"172.21.15.163:6831","nonce":9736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":9736},{"type":"v1","addr":"172.21.15.163:6829","nonce":9736}]},"public_addr":"172.21.15.163:6825/9736","cluster_addr":"172.21.15.163:6827/9736","heartbeat_back_addr":"172.21.15.163:6831/9736","heartbeat_front_addr":"172.21.15.163:6829/9736","state":["exists","up"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":9735},{"type":"v1","addr":"172.21.15.163:6817","nonce":9735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":9735},{"type":"v1","addr":"172.21.15.163:6819","nonce":9735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":9735},{"type":"v1","addr":"172.21.15.163:6824","nonce":9735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":9735},{"type":"v1","addr":"172.21.15.163:6821","nonce":9735}]},"public_addr":"172.21.15.163:6817/9735","cluster_addr":"172.21.15.163:6819/9735","heartbeat_back_addr":"172.21.15.163:6824/9735","heartbeat_front_addr":"172.21.15.163:6821/9735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.927198+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.495639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.042665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.255288+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.321453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.368712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.396738+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.392070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:33.979 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-11-29T11:13:33.979 INFO:tasks.ceph:Creating RBD pool 2019-11-29T11:13:33.979 INFO:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd pool create rbd 8 2019-11-29T11:13:35.329 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'rbd' created 2019-11-29T11:13:35.345 INFO:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-11-29T11:13:36.357 INFO:teuthology.orchestra.run.smithi069.stderr:enabled application 'rbd' on pool 'rbd' 2019-11-29T11:13:36.369 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-11-29T11:13:36.369 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:36.370 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-11-29T11:13:36.997 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:36.997 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:36.240321+0000","last_up_change":"2019-11-29T11:13:27.035374+0000","last_in_change":"2019-11-29T11:13:27.035374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-29T11:13:34.787825+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":9729},{"type":"v1","addr":"172.21.15.69:6811","nonce":9729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":9729},{"type":"v1","addr":"172.21.15.69:6813","nonce":9729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":9729},{"type":"v1","addr":"172.21.15.69:6817","nonce":9729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":9729},{"type":"v1","addr":"172.21.15.69:6815","nonce":9729}]},"public_addr":"172.21.15.69:6811/9729","cluster_addr":"172.21.15.69:6813/9729","heartbeat_back_addr":"172.21.15.69:6817/9729","heartbeat_front_addr":"172.21.15.69:6815/9729","state":["exists","up"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":9723},{"type":"v1","addr":"172.21.15.69:6803","nonce":9723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":9723},{"type":"v1","addr":"172.21.15.69:6805","nonce":9723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":9723},{"type":"v1","addr":"172.21.15.69:6809","nonce":9723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":9723},{"type":"v1","addr":"172.21.15.69:6807","nonce":9723}]},"public_addr":"172.21.15.69:6803/9723","cluster_addr":"172.21.15.69:6805/9723","heartbeat_back_addr":"172.21.15.69:6809/9723","heartbeat_front_addr":"172.21.15.69:6807/9723","state":["exists","up"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":9728},{"type":"v1","addr":"172.21.15.69:6819","nonce":9728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":9728},{"type":"v1","addr":"172.21.15.69:6821","nonce":9728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":9728},{"type":"v1","addr":"172.21.15.69:6825","nonce":9728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":9728},{"type":"v1","addr":"172.21.15.69:6823","nonce":9728}]},"public_addr":"172.21.15.69:6819/9728","cluster_addr":"172.21.15.69:6821/9728","heartbeat_back_addr":"172.21.15.69:6825/9728","heartbeat_front_addr":"172.21.15.69:6823/9728","state":["exists","up"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":9730},{"type":"v1","addr":"172.21.15.69:6827","nonce":9730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":9730},{"type":"v1","addr":"172.21.15.69:6829","nonce":9730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":9730},{"type":"v1","addr":"172.21.15.69:6833","nonce":9730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":9730},{"type":"v1","addr":"172.21.15.69:6831","nonce":9730}]},"public_addr":"172.21.15.69:6827/9730","cluster_addr":"172.21.15.69:6829/9730","heartbeat_back_addr":"172.21.15.69:6833/9730","heartbeat_front_addr":"172.21.15.69:6831/9730","state":["exists","up"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":9734},{"type":"v1","addr":"172.21.15.163:6801","nonce":9734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":9734},{"type":"v1","addr":"172.21.15.163:6803","nonce":9734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":9734},{"type":"v1","addr":"172.21.15.163:6807","nonce":9734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":9734},{"type":"v1","addr":"172.21.15.163:6805","nonce":9734}]},"public_addr":"172.21.15.163:6801/9734","cluster_addr":"172.21.15.163:6803/9734","heartbeat_back_addr":"172.21.15.163:6807/9734","heartbeat_front_addr":"172.21.15.163:6805/9734","state":["exists","up"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":9733},{"type":"v1","addr":"172.21.15.163:6809","nonce":9733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":9733},{"type":"v1","addr":"172.21.15.163:6811","nonce":9733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":9733},{"type":"v1","addr":"172.21.15.163:6815","nonce":9733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":9733},{"type":"v1","addr":"172.21.15.163:6813","nonce":9733}]},"public_addr":"172.21.15.163:6809/9733","cluster_addr":"172.21.15.163:6811/9733","heartbeat_back_addr":"172.21.15.163:6815/9733","heartbeat_front_addr":"172.21.15.163:6813/9733","state":["exists","up"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":9736},{"type":"v1","addr":"172.21.15.163:6825","nonce":9736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":9736},{"type":"v1","addr":"172.21.15.163:6827","nonce":9736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":9736},{"type":"v1","addr":"172.21.15.163:6831","nonce":9736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":9736},{"type":"v1","addr":"172.21.15.163:6829","nonce":9736}]},"public_addr":"172.21.15.163:6825/9736","cluster_addr":"172.21.15.163:6827/9736","heartbeat_back_addr":"172.21.15.163:6831/9736","heartbeat_front_addr":"172.21.15.163:6829/9736","state":["exists","up"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":9735},{"type":"v1","addr":"172.21.15.163:6817","nonce":9735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":9735},{"type":"v1","addr":"172.21.15.163:6819","nonce":9735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":9735},{"type":"v1","addr":"172.21.15.163:6824","nonce":9735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":9735},{"type":"v1","addr":"172.21.15.163:6821","nonce":9735}]},"public_addr":"172.21.15.163:6817/9735","cluster_addr":"172.21.15.163:6819/9735","heartbeat_back_addr":"172.21.15.163:6824/9735","heartbeat_front_addr":"172.21.15.163:6821/9735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.927198+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.495639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.042665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.255288+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.321453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.368712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.396738+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.392070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:37.014 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-11-29T11:13:34.787825+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'17', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-11-29T11:13:37.014 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-11-29T11:13:37.636 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 8 2019-11-29T11:13:37.648 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-11-29T11:13:37.649 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-11-29T11:13:37.649 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-11-29T11:13:38.483 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":8773},{"type":"v1","addr":"172.21.15.69:6801","nonce":8773}]},"active_addr":"172.21.15.69:6801/8773","active_change":"2019-11-29T11:13:04.290445+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"float","level":"advanced","flags":1,"default_value":"0.01","min":"0","max":"1","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the ratio between the fullest and least-full OSD is below this value then we stop trying to optimize placement.","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"float","level":"advanced","flags":1,"default_value":"0.01","min":"0","max":"1","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the ratio between the fullest and least-full OSD is below this value then we stop trying to optimize placement.","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-11-29T11:13:38.512 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-11-29T11:13:38.513 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-11-29T11:13:38.513 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:39.330 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:39.330 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":19,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:38.550009+0000","last_up_change":"2019-11-29T11:13:27.035374+0000","last_in_change":"2019-11-29T11:13:27.035374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-29T11:13:34.787825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":9729},{"type":"v1","addr":"172.21.15.69:6811","nonce":9729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":9729},{"type":"v1","addr":"172.21.15.69:6813","nonce":9729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":9729},{"type":"v1","addr":"172.21.15.69:6817","nonce":9729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":9729},{"type":"v1","addr":"172.21.15.69:6815","nonce":9729}]},"public_addr":"172.21.15.69:6811/9729","cluster_addr":"172.21.15.69:6813/9729","heartbeat_back_addr":"172.21.15.69:6817/9729","heartbeat_front_addr":"172.21.15.69:6815/9729","state":["exists","up"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":9723},{"type":"v1","addr":"172.21.15.69:6803","nonce":9723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":9723},{"type":"v1","addr":"172.21.15.69:6805","nonce":9723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":9723},{"type":"v1","addr":"172.21.15.69:6809","nonce":9723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":9723},{"type":"v1","addr":"172.21.15.69:6807","nonce":9723}]},"public_addr":"172.21.15.69:6803/9723","cluster_addr":"172.21.15.69:6805/9723","heartbeat_back_addr":"172.21.15.69:6809/9723","heartbeat_front_addr":"172.21.15.69:6807/9723","state":["exists","up"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":9728},{"type":"v1","addr":"172.21.15.69:6819","nonce":9728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":9728},{"type":"v1","addr":"172.21.15.69:6821","nonce":9728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":9728},{"type":"v1","addr":"172.21.15.69:6825","nonce":9728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":9728},{"type":"v1","addr":"172.21.15.69:6823","nonce":9728}]},"public_addr":"172.21.15.69:6819/9728","cluster_addr":"172.21.15.69:6821/9728","heartbeat_back_addr":"172.21.15.69:6825/9728","heartbeat_front_addr":"172.21.15.69:6823/9728","state":["exists","up"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":9730},{"type":"v1","addr":"172.21.15.69:6827","nonce":9730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":9730},{"type":"v1","addr":"172.21.15.69:6829","nonce":9730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":9730},{"type":"v1","addr":"172.21.15.69:6833","nonce":9730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":9730},{"type":"v1","addr":"172.21.15.69:6831","nonce":9730}]},"public_addr":"172.21.15.69:6827/9730","cluster_addr":"172.21.15.69:6829/9730","heartbeat_back_addr":"172.21.15.69:6833/9730","heartbeat_front_addr":"172.21.15.69:6831/9730","state":["exists","up"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":9734},{"type":"v1","addr":"172.21.15.163:6801","nonce":9734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":9734},{"type":"v1","addr":"172.21.15.163:6803","nonce":9734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":9734},{"type":"v1","addr":"172.21.15.163:6807","nonce":9734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":9734},{"type":"v1","addr":"172.21.15.163:6805","nonce":9734}]},"public_addr":"172.21.15.163:6801/9734","cluster_addr":"172.21.15.163:6803/9734","heartbeat_back_addr":"172.21.15.163:6807/9734","heartbeat_front_addr":"172.21.15.163:6805/9734","state":["exists","up"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":9733},{"type":"v1","addr":"172.21.15.163:6809","nonce":9733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":9733},{"type":"v1","addr":"172.21.15.163:6811","nonce":9733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":9733},{"type":"v1","addr":"172.21.15.163:6815","nonce":9733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":9733},{"type":"v1","addr":"172.21.15.163:6813","nonce":9733}]},"public_addr":"172.21.15.163:6809/9733","cluster_addr":"172.21.15.163:6811/9733","heartbeat_back_addr":"172.21.15.163:6815/9733","heartbeat_front_addr":"172.21.15.163:6813/9733","state":["exists","up"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":9736},{"type":"v1","addr":"172.21.15.163:6825","nonce":9736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":9736},{"type":"v1","addr":"172.21.15.163:6827","nonce":9736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":9736},{"type":"v1","addr":"172.21.15.163:6831","nonce":9736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":9736},{"type":"v1","addr":"172.21.15.163:6829","nonce":9736}]},"public_addr":"172.21.15.163:6825/9736","cluster_addr":"172.21.15.163:6827/9736","heartbeat_back_addr":"172.21.15.163:6831/9736","heartbeat_front_addr":"172.21.15.163:6829/9736","state":["exists","up"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":9735},{"type":"v1","addr":"172.21.15.163:6817","nonce":9735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":9735},{"type":"v1","addr":"172.21.15.163:6819","nonce":9735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":9735},{"type":"v1","addr":"172.21.15.163:6824","nonce":9735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":9735},{"type":"v1","addr":"172.21.15.163:6821","nonce":9735}]},"public_addr":"172.21.15.163:6817/9735","cluster_addr":"172.21.15.163:6819/9735","heartbeat_back_addr":"172.21.15.163:6824/9735","heartbeat_front_addr":"172.21.15.163:6821/9735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.927198+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.495639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.042665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.255288+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.321453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.368712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.396738+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.392070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:39.350 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-11-29T11:13:39.351 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-29T11:13:40.136 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:40.136 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":19,"fsid":"c38ae7c0-5696-449e-b9f9-b89969aec282","created":"2019-11-29T11:13:00.557880+0000","modified":"2019-11-29T11:13:38.550009+0000","last_up_change":"2019-11-29T11:13:27.035374+0000","last_in_change":"2019-11-29T11:13:27.035374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-29T11:13:34.787825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"7abe06a2-119e-4520-96ef-de89ca2fcee1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":9729},{"type":"v1","addr":"172.21.15.69:6811","nonce":9729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":9729},{"type":"v1","addr":"172.21.15.69:6813","nonce":9729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":9729},{"type":"v1","addr":"172.21.15.69:6817","nonce":9729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":9729},{"type":"v1","addr":"172.21.15.69:6815","nonce":9729}]},"public_addr":"172.21.15.69:6811/9729","cluster_addr":"172.21.15.69:6813/9729","heartbeat_back_addr":"172.21.15.69:6817/9729","heartbeat_front_addr":"172.21.15.69:6815/9729","state":["exists","up"]},{"osd":1,"uuid":"5850c501-9be4-4248-9e04-40790c9663ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":9723},{"type":"v1","addr":"172.21.15.69:6803","nonce":9723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":9723},{"type":"v1","addr":"172.21.15.69:6805","nonce":9723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":9723},{"type":"v1","addr":"172.21.15.69:6809","nonce":9723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":9723},{"type":"v1","addr":"172.21.15.69:6807","nonce":9723}]},"public_addr":"172.21.15.69:6803/9723","cluster_addr":"172.21.15.69:6805/9723","heartbeat_back_addr":"172.21.15.69:6809/9723","heartbeat_front_addr":"172.21.15.69:6807/9723","state":["exists","up"]},{"osd":2,"uuid":"1c233289-0e08-4505-9ecf-7aa1ffd4f544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":9728},{"type":"v1","addr":"172.21.15.69:6819","nonce":9728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":9728},{"type":"v1","addr":"172.21.15.69:6821","nonce":9728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":9728},{"type":"v1","addr":"172.21.15.69:6825","nonce":9728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":9728},{"type":"v1","addr":"172.21.15.69:6823","nonce":9728}]},"public_addr":"172.21.15.69:6819/9728","cluster_addr":"172.21.15.69:6821/9728","heartbeat_back_addr":"172.21.15.69:6825/9728","heartbeat_front_addr":"172.21.15.69:6823/9728","state":["exists","up"]},{"osd":3,"uuid":"cb5684dc-55be-4c85-973f-48ee3fa1e673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":9730},{"type":"v1","addr":"172.21.15.69:6827","nonce":9730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":9730},{"type":"v1","addr":"172.21.15.69:6829","nonce":9730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":9730},{"type":"v1","addr":"172.21.15.69:6833","nonce":9730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":9730},{"type":"v1","addr":"172.21.15.69:6831","nonce":9730}]},"public_addr":"172.21.15.69:6827/9730","cluster_addr":"172.21.15.69:6829/9730","heartbeat_back_addr":"172.21.15.69:6833/9730","heartbeat_front_addr":"172.21.15.69:6831/9730","state":["exists","up"]},{"osd":4,"uuid":"fbb2daaa-d3da-4ebd-a6b2-d447e50b2122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":9734},{"type":"v1","addr":"172.21.15.163:6801","nonce":9734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":9734},{"type":"v1","addr":"172.21.15.163:6803","nonce":9734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":9734},{"type":"v1","addr":"172.21.15.163:6807","nonce":9734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":9734},{"type":"v1","addr":"172.21.15.163:6805","nonce":9734}]},"public_addr":"172.21.15.163:6801/9734","cluster_addr":"172.21.15.163:6803/9734","heartbeat_back_addr":"172.21.15.163:6807/9734","heartbeat_front_addr":"172.21.15.163:6805/9734","state":["exists","up"]},{"osd":5,"uuid":"01c813d9-4c09-4c2c-a633-7b9065614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":9733},{"type":"v1","addr":"172.21.15.163:6809","nonce":9733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":9733},{"type":"v1","addr":"172.21.15.163:6811","nonce":9733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":9733},{"type":"v1","addr":"172.21.15.163:6815","nonce":9733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":9733},{"type":"v1","addr":"172.21.15.163:6813","nonce":9733}]},"public_addr":"172.21.15.163:6809/9733","cluster_addr":"172.21.15.163:6811/9733","heartbeat_back_addr":"172.21.15.163:6815/9733","heartbeat_front_addr":"172.21.15.163:6813/9733","state":["exists","up"]},{"osd":6,"uuid":"8401363e-0bb4-45ce-9f14-1d2c58ffec6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6823","nonce":9736},{"type":"v1","addr":"172.21.15.163:6825","nonce":9736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":9736},{"type":"v1","addr":"172.21.15.163:6827","nonce":9736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":9736},{"type":"v1","addr":"172.21.15.163:6831","nonce":9736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":9736},{"type":"v1","addr":"172.21.15.163:6829","nonce":9736}]},"public_addr":"172.21.15.163:6825/9736","cluster_addr":"172.21.15.163:6827/9736","heartbeat_back_addr":"172.21.15.163:6831/9736","heartbeat_front_addr":"172.21.15.163:6829/9736","state":["exists","up"]},{"osd":7,"uuid":"f5cfa113-2089-4f74-8db4-cdb1ab325122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":9735},{"type":"v1","addr":"172.21.15.163:6817","nonce":9735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":9735},{"type":"v1","addr":"172.21.15.163:6819","nonce":9735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":9735},{"type":"v1","addr":"172.21.15.163:6824","nonce":9735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":9735},{"type":"v1","addr":"172.21.15.163:6821","nonce":9735}]},"public_addr":"172.21.15.163:6817/9735","cluster_addr":"172.21.15.163:6819/9735","heartbeat_back_addr":"172.21.15.163:6824/9735","heartbeat_front_addr":"172.21.15.163:6821/9735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.927198+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.495639+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.042665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:24.255288+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.321453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.368712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.396738+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-29T11:13:23.392070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-29T11:13:40.156 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-11-29T11:13:40.762 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:40.763 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-11-29T11:13:41.309 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:41.309 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-11-29T11:13:41.859 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:41.860 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-11-29T11:13:42.522 INFO:teuthology.orchestra.run.smithi069.stdout:60129542149 2019-11-29T11:13:42.523 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-11-29T11:13:43.129 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:43.129 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-11-29T11:13:43.818 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:43.819 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-11-29T11:13:44.393 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:44.394 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-11-29T11:13:44.961 INFO:teuthology.orchestra.run.smithi069.stdout:55834574853 2019-11-29T11:13:44.961 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-11-29T11:13:45.809 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:45.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.0 2019-11-29T11:13:45.826 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-11-29T11:13:46.648 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:46.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.1 2019-11-29T11:13:46.672 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-11-29T11:13:47.590 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2019-11-29T11:13:47.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2019-11-29T11:13:47.606 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-11-29T11:13:48.391 INFO:teuthology.orchestra.run.smithi069.stdout:60129542150 2019-11-29T11:13:48.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.3 2019-11-29T11:13:48.405 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-11-29T11:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:49.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2019-11-29T11:13:49.225 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-11-29T11:13:50.111 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:50.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2019-11-29T11:13:50.113 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-11-29T11:13:50.873 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:50.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2019-11-29T11:13:50.889 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-11-29T11:13:51.707 INFO:teuthology.orchestra.run.smithi069.stdout:55834574854 2019-11-29T11:13:51.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2019-11-29T11:13:51.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-11-29T11:13:51.721 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-11-29T11:13:52.475 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2019-11-29T11:13:52.476 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:52.488 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-11-29T11:13:52.436913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699616,"kb_used_data":747895424,"kb_used_omap":299,"kb_used_meta":0,"kb_avail":747895424,"statfs":{"total":767585320960,"available":765844914176,"internally_reserved":0,"allocated":765844914176,"data_stored":765844914176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":306383,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.089890+0000","last_change":"2019-11-29T11:13:36.888585+0000","last_active":"2019-11-29T11:13:38.089890+0000","last_peered":"2019-11-29T11:13:38.089890+0000","last_clean":"2019-11-29T11:13:38.089890+0000","last_became_active":"2019-11-29T11:13:36.847357+0000","last_became_peered":"2019-11-29T11:13:36.847357+0000","last_unstale":"2019-11-29T11:13:38.089890+0000","last_undegraded":"2019-11-29T11:13:38.089890+0000","last_fullsized":"2019-11-29T11:13:38.089890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.078355+0000","last_change":"2019-11-29T11:13:36.982632+0000","last_active":"2019-11-29T11:13:38.078355+0000","last_peered":"2019-11-29T11:13:38.078355+0000","last_clean":"2019-11-29T11:13:38.078355+0000","last_became_active":"2019-11-29T11:13:36.972962+0000","last_became_peered":"2019-11-29T11:13:36.972962+0000","last_unstale":"2019-11-29T11:13:38.078355+0000","last_undegraded":"2019-11-29T11:13:38.078355+0000","last_fullsized":"2019-11-29T11:13:38.078355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:37.671759+0000","last_change":"2019-11-29T11:13:36.920914+0000","last_active":"2019-11-29T11:13:37.671759+0000","last_peered":"2019-11-29T11:13:37.671759+0000","last_clean":"2019-11-29T11:13:37.671759+0000","last_became_active":"2019-11-29T11:13:36.880800+0000","last_became_peered":"2019-11-29T11:13:36.880800+0000","last_unstale":"2019-11-29T11:13:37.671759+0000","last_undegraded":"2019-11-29T11:13:37.671759+0000","last_fullsized":"2019-11-29T11:13:37.671759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:37.653986+0000","last_change":"2019-11-29T11:13:36.783813+0000","last_active":"2019-11-29T11:13:37.653986+0000","last_peered":"2019-11-29T11:13:37.653986+0000","last_clean":"2019-11-29T11:13:37.653986+0000","last_became_active":"2019-11-29T11:13:36.693521+0000","last_became_peered":"2019-11-29T11:13:36.693521+0000","last_unstale":"2019-11-29T11:13:37.653986+0000","last_undegraded":"2019-11-29T11:13:37.653986+0000","last_fullsized":"2019-11-29T11:13:37.653986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.321695+0000","last_change":"2019-11-29T11:13:38.278526+0000","last_active":"2019-11-29T11:13:38.321695+0000","last_peered":"2019-11-29T11:13:38.321695+0000","last_clean":"2019-11-29T11:13:38.321695+0000","last_became_active":"2019-11-29T11:13:38.221455+0000","last_became_peered":"2019-11-29T11:13:38.221455+0000","last_unstale":"2019-11-29T11:13:38.321695+0000","last_undegraded":"2019-11-29T11:13:38.321695+0000","last_fullsized":"2019-11-29T11:13:38.321695+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.284444+0000","last_change":"2019-11-29T11:13:38.256152+0000","last_active":"2019-11-29T11:13:38.284444+0000","last_peered":"2019-11-29T11:13:38.284444+0000","last_clean":"2019-11-29T11:13:38.284444+0000","last_became_active":"2019-11-29T11:13:38.212098+0000","last_became_peered":"2019-11-29T11:13:38.212098+0000","last_unstale":"2019-11-29T11:13:38.284444+0000","last_undegraded":"2019-11-29T11:13:38.284444+0000","last_fullsized":"2019-11-29T11:13:38.284444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.325270+0000","last_change":"2019-11-29T11:13:38.302309+0000","last_active":"2019-11-29T11:13:38.325270+0000","last_peered":"2019-11-29T11:13:38.325270+0000","last_clean":"2019-11-29T11:13:38.325270+0000","last_became_active":"2019-11-29T11:13:38.293312+0000","last_became_peered":"2019-11-29T11:13:38.293312+0000","last_unstale":"2019-11-29T11:13:38.325270+0000","last_undegraded":"2019-11-29T11:13:38.325270+0000","last_fullsized":"2019-11-29T11:13:38.325270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.292000+0000","last_change":"2019-11-29T11:13:38.265539+0000","last_active":"2019-11-29T11:13:38.292000+0000","last_peered":"2019-11-29T11:13:38.292000+0000","last_clean":"2019-11-29T11:13:38.292000+0000","last_became_active":"2019-11-29T11:13:38.231031+0000","last_became_peered":"2019-11-29T11:13:38.231031+0000","last_unstale":"2019-11-29T11:13:38.292000+0000","last_undegraded":"2019-11-29T11:13:38.292000+0000","last_fullsized":"2019-11-29T11:13:38.292000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37692,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.419}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.892}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.242}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.687}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.717}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":206.020}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.699}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37793,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.623}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.864}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":49.411}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.116}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.542}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.241}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.598}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.388}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.816}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.085}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.926}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.342}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.852}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.215}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.096}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.416}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.362}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.457}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.092}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32891,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.421}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.198}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.060}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42197,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.068}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.644}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.578}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.862}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.437}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":44.658}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47333,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.587}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.142}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.233}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.941}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.213}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.567}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42607,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.820}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.508}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.210}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.612}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.682}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.040}]}]}]}} 2019-11-29T11:13:52.490 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-11-29T11:13:53.188 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2019-11-29T11:13:53.189 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:53.202 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-11-29T11:13:52.436913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699616,"kb_used_data":747895424,"kb_used_omap":299,"kb_used_meta":0,"kb_avail":747895424,"statfs":{"total":767585320960,"available":765844914176,"internally_reserved":0,"allocated":765844914176,"data_stored":765844914176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":306383,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.089890+0000","last_change":"2019-11-29T11:13:36.888585+0000","last_active":"2019-11-29T11:13:38.089890+0000","last_peered":"2019-11-29T11:13:38.089890+0000","last_clean":"2019-11-29T11:13:38.089890+0000","last_became_active":"2019-11-29T11:13:36.847357+0000","last_became_peered":"2019-11-29T11:13:36.847357+0000","last_unstale":"2019-11-29T11:13:38.089890+0000","last_undegraded":"2019-11-29T11:13:38.089890+0000","last_fullsized":"2019-11-29T11:13:38.089890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.078355+0000","last_change":"2019-11-29T11:13:36.982632+0000","last_active":"2019-11-29T11:13:38.078355+0000","last_peered":"2019-11-29T11:13:38.078355+0000","last_clean":"2019-11-29T11:13:38.078355+0000","last_became_active":"2019-11-29T11:13:36.972962+0000","last_became_peered":"2019-11-29T11:13:36.972962+0000","last_unstale":"2019-11-29T11:13:38.078355+0000","last_undegraded":"2019-11-29T11:13:38.078355+0000","last_fullsized":"2019-11-29T11:13:38.078355+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:37.671759+0000","last_change":"2019-11-29T11:13:36.920914+0000","last_active":"2019-11-29T11:13:37.671759+0000","last_peered":"2019-11-29T11:13:37.671759+0000","last_clean":"2019-11-29T11:13:37.671759+0000","last_became_active":"2019-11-29T11:13:36.880800+0000","last_became_peered":"2019-11-29T11:13:36.880800+0000","last_unstale":"2019-11-29T11:13:37.671759+0000","last_undegraded":"2019-11-29T11:13:37.671759+0000","last_fullsized":"2019-11-29T11:13:37.671759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:37.653986+0000","last_change":"2019-11-29T11:13:36.783813+0000","last_active":"2019-11-29T11:13:37.653986+0000","last_peered":"2019-11-29T11:13:37.653986+0000","last_clean":"2019-11-29T11:13:37.653986+0000","last_became_active":"2019-11-29T11:13:36.693521+0000","last_became_peered":"2019-11-29T11:13:36.693521+0000","last_unstale":"2019-11-29T11:13:37.653986+0000","last_undegraded":"2019-11-29T11:13:37.653986+0000","last_fullsized":"2019-11-29T11:13:37.653986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.321695+0000","last_change":"2019-11-29T11:13:38.278526+0000","last_active":"2019-11-29T11:13:38.321695+0000","last_peered":"2019-11-29T11:13:38.321695+0000","last_clean":"2019-11-29T11:13:38.321695+0000","last_became_active":"2019-11-29T11:13:38.221455+0000","last_became_peered":"2019-11-29T11:13:38.221455+0000","last_unstale":"2019-11-29T11:13:38.321695+0000","last_undegraded":"2019-11-29T11:13:38.321695+0000","last_fullsized":"2019-11-29T11:13:38.321695+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.284444+0000","last_change":"2019-11-29T11:13:38.256152+0000","last_active":"2019-11-29T11:13:38.284444+0000","last_peered":"2019-11-29T11:13:38.284444+0000","last_clean":"2019-11-29T11:13:38.284444+0000","last_became_active":"2019-11-29T11:13:38.212098+0000","last_became_peered":"2019-11-29T11:13:38.212098+0000","last_unstale":"2019-11-29T11:13:38.284444+0000","last_undegraded":"2019-11-29T11:13:38.284444+0000","last_fullsized":"2019-11-29T11:13:38.284444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.325270+0000","last_change":"2019-11-29T11:13:38.302309+0000","last_active":"2019-11-29T11:13:38.325270+0000","last_peered":"2019-11-29T11:13:38.325270+0000","last_clean":"2019-11-29T11:13:38.325270+0000","last_became_active":"2019-11-29T11:13:38.293312+0000","last_became_peered":"2019-11-29T11:13:38.293312+0000","last_unstale":"2019-11-29T11:13:38.325270+0000","last_undegraded":"2019-11-29T11:13:38.325270+0000","last_fullsized":"2019-11-29T11:13:38.325270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-11-29T11:13:38.292000+0000","last_change":"2019-11-29T11:13:38.265539+0000","last_active":"2019-11-29T11:13:38.292000+0000","last_peered":"2019-11-29T11:13:38.292000+0000","last_clean":"2019-11-29T11:13:38.292000+0000","last_became_active":"2019-11-29T11:13:38.231031+0000","last_became_peered":"2019-11-29T11:13:38.231031+0000","last_unstale":"2019-11-29T11:13:38.292000+0000","last_undegraded":"2019-11-29T11:13:38.292000+0000","last_fullsized":"2019-11-29T11:13:38.292000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-29T11:13:34.894149+0000","last_clean_scrub_stamp":"2019-11-29T11:13:34.894149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37692,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.419}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.892}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.242}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.687}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.717}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":206.020}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.699}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37793,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.623}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.864}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":49.411}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.116}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.542}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.241}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.598}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.388}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.816}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.085}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.926}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.342}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.852}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.215}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.096}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.416}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.362}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.457}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.092}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32891,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.421}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.198}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.060}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42197,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.068}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.644}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.578}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.862}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.437}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":44.658}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47333,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.587}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.142}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.233}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.941}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.213}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.567}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212452,"kb_used_data":93486928,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486928,"statfs":{"total":95948165120,"available":95730614272,"internally_reserved":0,"allocated":95730614272,"data_stored":95730614272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42607,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.820}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.508}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.210}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.612}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.682}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.040}]}]}]}} 2019-11-29T11:13:53.204 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-11-29T11:13:53.204 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-11-29T11:13:53.204 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-11-29T11:13:53.205 INFO:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-11-29T11:13:54.111 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:13:54.137 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-11-29T11:13:54.137 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-11-29T11:13:54.138 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-11-29T11:13:54.167 INFO:teuthology.run_tasks:Running task rgw... 2019-11-29T11:13:54.196 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-11-29T11:13:54.196 DEBUG:tasks.rgw:client list is ['client.0'] 2019-11-29T11:13:54.197 INFO:tasks.rgw:Creating data pools 2019-11-29T11:13:54.197 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-11-29T11:13:54.197 INFO:teuthology.orchestra.run.smithi069:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-11-29T11:13:56.181 INFO:teuthology.orchestra.run.smithi069:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-11-29T11:13:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'default.rgw.buckets.data' created 2019-11-29T11:13:58.774 INFO:teuthology.orchestra.run.smithi069:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-11-29T11:13:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-11-29T11:13:59.753 INFO:teuthology.orchestra.run.smithi069:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-11-29T11:14:00.948 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'default.rgw.buckets.index' created 2019-11-29T11:14:00.964 INFO:teuthology.orchestra.run.smithi069:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-11-29T11:14:02.062 INFO:teuthology.orchestra.run.smithi069.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-11-29T11:14:02.087 DEBUG:tasks.rgw:Pools created 2019-11-29T11:14:02.087 INFO:tasks.rgw:Configuring compression type = random 2019-11-29T11:14:02.088 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-11-29T11:14:02.088 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-11-29T11:14:02.088 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-11-29T11:14:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:02.275+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:04.937+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:04.913 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:04.937+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:05.013 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:05.030+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:05.013 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:05.030+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:05.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:05.041+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:05.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:05.041+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:05.047+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.265+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.273+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.273+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.281+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.281+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.264 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.288+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.264 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.288+0000 7f606070eec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:14:06.264 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.288+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.295+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.295+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.302+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:06.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.302+0000 7f606070eec0 10 Creating default zonegroup 2019-11-29T11:14:06.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.302+0000 7f606070eec0 20 get_system_obj_state: rctx=0x7fffe6b57b20 obj=:.pools.avail state=0x55cc131814c0 s->prefetch_data=0 2019-11-29T11:14:06.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.304+0000 7f606070eec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-11-29T11:14:06.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.493+0000 7f606070eec0 10 WARNING: store->list_zones() returned r=-2 2019-11-29T11:14:06.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.493+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:06.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:06.500+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:07.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.610+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:07.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.618+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:07.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.618+0000 7f606070eec0 10 could not read realm id: (2) No such file or directory 2019-11-29T11:14:07.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.618+0000 7f606070eec0 10 WARNING: failed to set zone as default, r=-22 2019-11-29T11:14:07.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.618+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.625+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:07.938 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:07.963+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:07.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.011+0000 7f606070eec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-29T11:14:07.987 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.011+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:08.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.030+0000 7f606070eec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-29T11:14:08.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.030+0000 7f606070eec0 20 zonegroup default 2019-11-29T11:14:08.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.030+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:08.015 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.039+0000 7f606070eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:08.015 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.039+0000 7f606070eec0 10 Cannot find current period zone using local zone 2019-11-29T11:14:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.039+0000 7f606070eec0 10 Using default name default 2019-11-29T11:14:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.040+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:08.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.092+0000 7f606070eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:08.069 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.093+0000 7f606070eec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:08.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.111+0000 7f606070eec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-29T11:14:08.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.111+0000 7f606070eec0 20 zone default found 2019-11-29T11:14:08.088 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.111+0000 7f606070eec0 20 started sync module instance, tier type = 2019-11-29T11:14:08.088 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.111+0000 7f606070eec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:14:08.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:08.116+0000 7f604e13c700 0 -- 172.21.15.69:0/1050591966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cc1316a350 msgr2=0x55cc13171cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:14:11.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:11.516+0000 7f604d93b700 0 -- 172.21.15.69:0/1050591966 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x55cc1317e4e0 msgr2=0x55cc13180900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T11:14:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=0 2019-11-29T11:14:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=1 2019-11-29T11:14:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=2 2019-11-29T11:14:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=3 2019-11-29T11:14:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=4 2019-11-29T11:14:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=5 2019-11-29T11:14:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.849+0000 7f606070eec0 20 add_watcher() i=6 2019-11-29T11:14:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.891+0000 7f606070eec0 20 add_watcher() i=7 2019-11-29T11:14:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.891+0000 7f606070eec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:14:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:12.892+0000 7f601e7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:14:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:17.010+0000 7f606070eec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-11-29T11:14:21.320 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:21.345+0000 7f606070eec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-11-29T11:14:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:24.753+0000 7f5f8cff9700 20 reqs_thread_entry: start 2019-11-29T11:14:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:24.753+0000 7f5f7e7fc700 20 reqs_thread_entry: start 2019-11-29T11:14:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:24.753+0000 7f606070eec0 20 init_complete bucket index max shards: 0 2019-11-29T11:14:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:24.753+0000 7f5f7cff9700 20 reqs_thread_entry: start 2019-11-29T11:14:25.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.188+0000 7f606070eec0 20 remove_watcher() i=0 2019-11-29T11:14:25.164 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.188+0000 7f606070eec0 2 removed watcher, disabling cache 2019-11-29T11:14:25.259 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.284+0000 7f606070eec0 20 remove_watcher() i=1 2019-11-29T11:14:25.337 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.361+0000 7f606070eec0 20 remove_watcher() i=2 2019-11-29T11:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.459+0000 7f606070eec0 20 remove_watcher() i=3 2019-11-29T11:14:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.559+0000 7f606070eec0 20 remove_watcher() i=4 2019-11-29T11:14:25.610 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.634+0000 7f606070eec0 20 remove_watcher() i=5 2019-11-29T11:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.719+0000 7f606070eec0 20 remove_watcher() i=6 2019-11-29T11:14:25.764 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:25.789+0000 7f606070eec0 20 remove_watcher() i=7 2019-11-29T11:14:25.777 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2019-11-29T11:14:25.778 DEBUG:tasks.util.rgw: json result: [] 2019-11-29T11:14:25.778 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-11-29T11:14:25.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-11-29T11:14:25.778 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-11-29T11:14:26.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.026+0000 7f8fa4ff9700 20 reqs_thread_entry: start 2019-11-29T11:14:26.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.026+0000 7f906798aec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.039+0000 7f906798aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.039+0000 7f906798aec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.039 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.061+0000 7f906798aec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-29T11:14:26.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.062+0000 7f906798aec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.067+0000 7f906798aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.067+0000 7f906798aec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.063 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.082+0000 7f906798aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.082+0000 7f906798aec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.094+0000 7f906798aec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-29T11:14:26.190 INFO:teuthology.orchestra.run.smithi069.stdout:{"id":"42bb32be-1e05-491f-815b-a606bdf24d77","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-29T11:14:26.190 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'42bb32be-1e05-491f-815b-a606bdf24d77', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-29T11:14:26.190 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-11-29T11:14:26.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-11-29T11:14:26.191 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-11-29T11:14:26.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.380+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.391 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.413+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.391 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.413+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.421+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.421+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.426+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.427+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.435+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.635 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.659+0000 7fd007c0bec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:14:26.635 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.659+0000 7fd007c0bec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:14:26.638 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.659+0000 7fd007c0bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:14:26.638 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.659+0000 7fd007c0bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:14:26.638 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.659+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.666+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.666+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.679+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-29T11:14:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.679+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.687+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.687+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.699+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-29T11:14:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.699+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.708+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.708+0000 7fd007c0bec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:14:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.708+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.714+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.714+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.723+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.723+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.738+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-29T11:14:26.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.738+0000 7fd007c0bec0 20 zonegroup default 2019-11-29T11:14:26.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.738+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.720 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.745+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.721 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.745+0000 7fd007c0bec0 10 Cannot find current period zone using local zone 2019-11-29T11:14:26.721 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.745+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.751+0000 7fd007c0bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:26.731 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.751+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.765+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:26.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.765+0000 7fd007c0bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.782+0000 7fd007c0bec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-29T11:14:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.782+0000 7fd007c0bec0 20 zone default found 2019-11-29T11:14:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.782+0000 7fd007c0bec0 20 started sync module instance, tier type = 2019-11-29T11:14:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:26.782+0000 7fd007c0bec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:14:27.220 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=0 2019-11-29T11:14:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=1 2019-11-29T11:14:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=2 2019-11-29T11:14:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=3 2019-11-29T11:14:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=4 2019-11-29T11:14:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.244+0000 7fd007c0bec0 20 add_watcher() i=5 2019-11-29T11:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.272+0000 7fd007c0bec0 20 add_watcher() i=6 2019-11-29T11:14:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.299+0000 7fd007c0bec0 20 add_watcher() i=7 2019-11-29T11:14:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.299+0000 7fd007c0bec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:14:27.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.300+0000 7fcfc57e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:14:27.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.302+0000 7fcff5639700 0 -- 172.21.15.69:0/851717274 >> v1:172.21.15.163:6789/0 conn(0x56416f158b30 legacy=0x56416f15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-29T11:14:27.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:27.357+0000 7fcff5639700 0 -- 172.21.15.69:0/3712214877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56416f15eb90 msgr2=0x56416f15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:14:29.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:29.866+0000 7fcf377fe700 20 reqs_thread_entry: start 2019-11-29T11:14:29.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:29.866+0000 7fcf357fa700 20 reqs_thread_entry: start 2019-11-29T11:14:29.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:29.866+0000 7fd007c0bec0 20 init_complete bucket index max shards: 0 2019-11-29T11:14:29.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:29.867+0000 7fcf237fe700 20 reqs_thread_entry: start 2019-11-29T11:14:30.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.166+0000 7fd007c0bec0 20 remove_watcher() i=0 2019-11-29T11:14:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.166+0000 7fd007c0bec0 2 removed watcher, disabling cache 2019-11-29T11:14:30.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.232+0000 7fd007c0bec0 20 remove_watcher() i=1 2019-11-29T11:14:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.290+0000 7fd007c0bec0 20 remove_watcher() i=2 2019-11-29T11:14:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.357+0000 7fd007c0bec0 20 remove_watcher() i=3 2019-11-29T11:14:30.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.425+0000 7fd007c0bec0 20 remove_watcher() i=4 2019-11-29T11:14:30.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.483+0000 7fd007c0bec0 20 remove_watcher() i=5 2019-11-29T11:14:30.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.550+0000 7fd007c0bec0 20 remove_watcher() i=6 2019-11-29T11:14:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.623+0000 7fd007c0bec0 20 remove_watcher() i=7 2019-11-29T11:14:30.609 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2019-11-29T11:14:30.609 DEBUG:tasks.util.rgw: json result: [] 2019-11-29T11:14:30.609 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-11-29T11:14:30.610 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-11-29T11:14:30.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-11-29T11:14:30.610 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-11-29T11:14:30.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.831+0000 7f0adffff700 20 reqs_thread_entry: start 2019-11-29T11:14:30.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.831+0000 7f0bab18cec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:30.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.843+0000 7f0bab18cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.843+0000 7f0bab18cec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:30.827 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.852+0000 7f0bab18cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:30.827 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.852+0000 7f0bab18cec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.879+0000 7f0bab18cec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-29T11:14:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.879+0000 7f0bab18cec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:30.898+0000 7f0bab18cec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-29T11:14:30.978 INFO:teuthology.orchestra.run.smithi069.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-11-29T11:14:30.978 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-11-29T11:14:30.979 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-11-29T11:14:30.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-11-29T11:14:30.979 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-11-29T11:14:31.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.205+0000 7f7233fff700 20 reqs_thread_entry: start 2019-11-29T11:14:31.181 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.205+0000 7f72ff0c6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:31.192 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.216+0000 7f72ff0c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:31.192 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.216+0000 7f72ff0c6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:31.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.246+0000 7f72ff0c6ec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-29T11:14:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.246+0000 7f72ff0c6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.250+0000 7f72ff0c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:31.232 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.250+0000 7f72ff0c6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:31.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.261+0000 7f72ff0c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:31.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.261+0000 7f72ff0c6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:31.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:31.282+0000 7f72ff0c6ec0 20 rados_obj.operate() r=0 bl.length=372 2019-11-29T11:14:31.343 INFO:teuthology.orchestra.run.smithi069.stdout:{"id":"42bb32be-1e05-491f-815b-a606bdf24d77","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-29T11:14:31.984 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'42bb32be-1e05-491f-815b-a606bdf24d77', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-29T11:14:31.985 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-11-29T11:14:31.985 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-11-29T11:14:31.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-11-29T11:14:31.985 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-11-29T11:14:32.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.190+0000 7eff99ffb700 20 reqs_thread_entry: start 2019-11-29T11:14:32.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.191+0000 7f0064cbeec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.199+0000 7f0064cbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.199+0000 7f0064cbeec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.206+0000 7f0064cbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.206+0000 7f0064cbeec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.204 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.229+0000 7f0064cbeec0 20 rados_obj.operate() r=0 bl.length=372 2019-11-29T11:14:32.204 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.229+0000 7f0064cbeec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.243+0000 7f0064cbeec0 20 rados_obj.operate() r=0 bl.length=888 2019-11-29T11:14:32.271 INFO:teuthology.orchestra.run.smithi069.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-11-29T11:14:32.271 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-11-29T11:14:32.271 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-11-29T11:14:32.271 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-11-29T11:14:32.272 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-11-29T11:14:32.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.501+0000 7fc15affd700 20 reqs_thread_entry: start 2019-11-29T11:14:32.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.501+0000 7fc22593bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.530+0000 7fc22593bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:32.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.530+0000 7fc22593bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.547+0000 7fc22593bec0 20 rados_obj.operate() r=0 bl.length=888 2019-11-29T11:14:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.547+0000 7fc22593bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.551+0000 7fc22593bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:14:32.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.552+0000 7fc22593bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.561+0000 7fc22593bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:14:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.561+0000 7fc22593bec0 20 rados->read ofs=0 len=0 2019-11-29T11:14:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:14:32.570+0000 7fc22593bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:14:32.699 INFO:teuthology.orchestra.run.smithi069.stdout:{"id":"42bb32be-1e05-491f-815b-a606bdf24d77","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-29T11:14:32.699 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'42bb32be-1e05-491f-815b-a606bdf24d77', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-29T11:14:32.699 INFO:tasks.rgw:Starting rgw... 2019-11-29T11:14:32.700 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-11-29T11:14:32.700 INFO:tasks.rgw:Using civetweb as radosgw frontend 2019-11-29T11:14:32.700 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-11-29T11:14:32.700 INFO:tasks.rgw.client.0:Restarting daemon 2019-11-29T11:14:32.700 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-11-29T11:14:32.704 INFO:tasks.rgw.client.0:Started 2019-11-29T11:14:32.705 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:32.705 INFO:teuthology.orchestra.run.smithi069:> curl http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:32.778 INFO:teuthology.orchestra.run.smithi069.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-29T11:14:32.778 INFO:teuthology.orchestra.run.smithi069.stderr: Dload Upload Total Spent Left Speed 2019-11-29T11:14:32.789 INFO:teuthology.orchestra.run.smithi069.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi069.front.sepia.ceph.com:80; Connection refused 2019-11-29T11:14:32.790 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-29T11:14:33.791 INFO:teuthology.orchestra.run.smithi069:> curl http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:33.879 INFO:teuthology.orchestra.run.smithi069.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-29T11:14:33.879 INFO:teuthology.orchestra.run.smithi069.stderr: Dload Upload Total Spent Left Speed 2019-11-29T11:14:33.887 INFO:teuthology.orchestra.run.smithi069.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi069.front.sepia.ceph.com:80; Connection refused 2019-11-29T11:14:33.887 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-29T11:14:35.888 INFO:teuthology.orchestra.run.smithi069:> curl http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:35.981 INFO:teuthology.orchestra.run.smithi069.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-29T11:14:35.982 INFO:teuthology.orchestra.run.smithi069.stderr: Dload Upload Total Spent Left Speed 2019-11-29T11:14:35.985 INFO:teuthology.orchestra.run.smithi069.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi069.front.sepia.ceph.com:80; Connection refused 2019-11-29T11:14:35.985 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-29T11:14:39.987 INFO:teuthology.orchestra.run.smithi069:> curl http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:40.078 INFO:teuthology.orchestra.run.smithi069.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-29T11:14:40.079 INFO:teuthology.orchestra.run.smithi069.stderr: Dload Upload Total Spent Left Speed 2019-11-29T11:14:40.082 INFO:teuthology.orchestra.run.smithi069.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi069.front.sepia.ceph.com:80; Connection refused 2019-11-29T11:14:40.083 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-29T11:14:48.084 INFO:teuthology.orchestra.run.smithi069:> curl http://smithi069.front.sepia.ceph.com:80/ 2019-11-29T11:14:48.154 INFO:teuthology.orchestra.run.smithi069.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-29T11:14:48.156 INFO:teuthology.orchestra.run.smithi069.stderr: Dload Upload Total Spent Left Speed 2019-11-29T11:14:48.703 INFO:teuthology.orchestra.run.smithi069.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 389 0 --:--:-- --:--:-- --:--:-- 389 2019-11-29T11:14:48.703 INFO:teuthology.orchestra.run.smithi069.stdout:anonymous 2019-11-29T11:14:48.704 INFO:teuthology.run_tasks:Running task workunit... 2019-11-29T11:14:48.732 INFO:tasks.workunit:Pulling workunits from ref 656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:14:48.732 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-11-29T11:14:48.732 INFO:teuthology.orchestra.run.smithi069:> stat -- /home/ubuntu/cephtest/mnt.0 2019-11-29T11:14:48.784 INFO:teuthology.orchestra.run.smithi069.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-11-29T11:14:48.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-29T11:14:48.785 INFO:teuthology.orchestra.run.smithi069:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-11-29T11:14:48.837 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-11-29T11:14:48.837 INFO:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-11-29T11:14:48.902 INFO:tasks.workunit:timeout=3h 2019-11-29T11:14:48.902 INFO:tasks.workunit:cleanup=True 2019-11-29T11:14:48.903 INFO:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 656c8e8049c2c1acd363143c842c2edf1fe09b64 2019-11-29T11:14:49.204 INFO:tasks.workunit.client.0.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-11-29T11:15:54.471 INFO:tasks.workunit.client.0.smithi069.stderr:Note: checking out '656c8e8049c2c1acd363143c842c2edf1fe09b64'. 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr: 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr:state without impacting any branches by performing another checkout. 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr: 2019-11-29T11:15:54.472 INFO:tasks.workunit.client.0.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2019-11-29T11:15:54.473 INFO:tasks.workunit.client.0.smithi069.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-11-29T11:15:54.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2019-11-29T11:15:54.473 INFO:tasks.workunit.client.0.smithi069.stderr: git checkout -b new_branch_name 2019-11-29T11:15:54.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2019-11-29T11:15:54.473 INFO:tasks.workunit.client.0.smithi069.stderr:HEAD is now at 656c8e8... Merge pull request #31856 from rzarzynski/wip-crimson-errorator-handle_error 2019-11-29T11:15:54.474 INFO:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-11-29T11:15:54.526 INFO:tasks.workunit.client.0.smithi069.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-11-29T11:15:54.528 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-11-29T11:15:54.528 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-11-29T11:15:54.996 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-11-29T11:15:55.094 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-11-29T11:15:55.148 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-11-29T11:15:55.149 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-11-29T11:15:55.149 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-11-29T11:15:55.181 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-11-29T11:15:55.238 DEBUG:teuthology.orchestra.remote:smithi069:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-11-29T11:15:55.293 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-11-29T11:15:55.294 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-11-29T11:15:55.294 INFO:teuthology.orchestra.run.smithi069:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=656c8e8049c2c1acd363143c842c2edf1fe09b64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-11-29T11:15:55.350 INFO:tasks.workunit.client.0.smithi069.stdout:Running main() from gmock_main.cc 2019-11-29T11:15:55.350 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] Running 11 tests from 1 test suite. 2019-11-29T11:15:55.351 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment set-up. 2019-11-29T11:15:55.351 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 11 tests from ClsLock 2019-11-29T11:15:55.352 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-11-29T11:15:57.753 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:15:57.754 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: exclusive 2019-11-29T11:15:57.754 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:15:57.754 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:15:57.754 INFO:tasks.workunit.client.0.smithi069.stdout: client.4539 expiration=2019-11-29T11:17:57.638326+0000 addr=v1:172.21.15.69:0/4027408772 cookie= 2019-11-29T11:15:57.902 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:15:57.903 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: shared 2019-11-29T11:15:57.903 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:15:57.903 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:15:57.903 INFO:tasks.workunit.client.0.smithi069.stdout: client.4361 expiration=0.000000 addr=v1:172.21.15.69:0/1835945287 cookie= 2019-11-29T11:15:57.903 INFO:tasks.workunit.client.0.smithi069.stdout: client.4539 expiration=2019-11-29T11:17:57.884498+0000 addr=v1:172.21.15.69:0/4027408772 cookie= 2019-11-29T11:15:57.949 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:15:57.950 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: shared 2019-11-29T11:15:57.950 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:15:57.950 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:15:57.951 INFO:tasks.workunit.client.0.smithi069.stdout: client.4361 expiration=0.000000 addr=v1:172.21.15.69:0/1835945287 cookie= 2019-11-29T11:15:59.129 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestMultiLocking (3779 ms) 2019-11-29T11:15:59.129 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestMeta 2019-11-29T11:16:02.215 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:16:02.216 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: shared 2019-11-29T11:16:02.216 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:16:02.216 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:16:02.216 INFO:tasks.workunit.client.0.smithi069.stdout: client.4480 expiration=0.000000 addr=v1:172.21.15.69:0/306083333 cookie= 2019-11-29T11:16:02.304 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:16:02.305 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: exclusive 2019-11-29T11:16:02.305 INFO:tasks.workunit.client.0.smithi069.stdout: tag: new_tag 2019-11-29T11:16:02.305 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:16:02.305 INFO:tasks.workunit.client.0.smithi069.stdout: client.4367 expiration=0.000000 addr=v1:172.21.15.69:0/1488048470 cookie= 2019-11-29T11:16:03.511 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestMeta (4382 ms) 2019-11-29T11:16:03.511 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestCookie 2019-11-29T11:16:06.485 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:16:06.485 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: exclusive 2019-11-29T11:16:06.485 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:16:06.485 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:16:06.554 INFO:tasks.workunit.client.0.smithi069.stdout:lock: mylock 2019-11-29T11:16:06.554 INFO:tasks.workunit.client.0.smithi069.stdout: lock_type: shared 2019-11-29T11:16:06.554 INFO:tasks.workunit.client.0.smithi069.stdout: tag: 2019-11-29T11:16:06.554 INFO:tasks.workunit.client.0.smithi069.stdout: lockers: 2019-11-29T11:16:06.555 INFO:tasks.workunit.client.0.smithi069.stdout: client.4557 expiration=0.000000 addr=v1:172.21.15.69:0/3941255593 cookie= 2019-11-29T11:16:06.555 INFO:tasks.workunit.client.0.smithi069.stdout: client.4557 expiration=0.000000 addr=v1:172.21.15.69:0/3941255593 cookie=new cookie 2019-11-29T11:16:06.993 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestCookie (3479 ms) 2019-11-29T11:16:06.993 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-11-29T11:16:10.510 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestMultipleLocks (3520 ms) 2019-11-29T11:16:10.510 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestLockDuration 2019-11-29T11:16:19.059 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestLockDuration (8550 ms) 2019-11-29T11:16:19.060 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-11-29T11:16:22.526 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestAssertLocked (3467 ms) 2019-11-29T11:16:22.526 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestSetCookie 2019-11-29T11:16:26.088 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestSetCookie (3563 ms) 2019-11-29T11:16:26.088 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestRenew 2019-11-29T11:16:47.719 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestRenew (21633 ms) 2019-11-29T11:16:47.720 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-11-29T11:16:54.642 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6922 ms) 2019-11-29T11:16:54.642 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-11-29T11:17:02.626 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7985 ms) 2019-11-29T11:17:02.626 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-11-29T11:17:10.751 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8126 ms) 2019-11-29T11:17:10.751 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 11 tests from ClsLock (75406 ms total) 2019-11-29T11:17:10.751 INFO:tasks.workunit.client.0.smithi069.stdout: 2019-11-29T11:17:10.752 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment tear-down 2019-11-29T11:17:10.752 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] 11 tests from 1 test suite ran. (75406 ms total) 2019-11-29T11:17:10.752 INFO:tasks.workunit.client.0.smithi069.stdout:[ PASSED ] 11 tests. 2019-11-29T11:17:10.753 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-29T11:17:10.753 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-29T11:17:10.953 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-11-29T11:17:10.954 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-11-29T11:17:10.954 INFO:teuthology.orchestra.run.smithi069:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=656c8e8049c2c1acd363143c842c2edf1fe09b64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-11-29T11:17:11.055 INFO:tasks.workunit.client.0.smithi069.stdout:Running main() from gmock_main.cc 2019-11-29T11:17:11.055 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] Running 4 tests from 1 test suite. 2019-11-29T11:17:11.055 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment set-up. 2019-11-29T11:17:11.056 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 4 tests from cls_log 2019-11-29T11:17:11.056 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-11-29T11:17:14.232 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_log.test_log_add_same_time (3177 ms) 2019-11-29T11:17:14.232 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-11-29T11:17:18.783 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_log.test_log_add_different_time (4552 ms) 2019-11-29T11:17:18.783 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_log.trim_by_time 2019-11-29T11:17:24.539 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_log.trim_by_time (5756 ms) 2019-11-29T11:17:24.539 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_log.trim_by_marker 2019-11-29T11:17:28.979 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_log.trim_by_marker (4441 ms) 2019-11-29T11:17:28.979 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 4 tests from cls_log (17926 ms total) 2019-11-29T11:17:28.979 INFO:tasks.workunit.client.0.smithi069.stdout: 2019-11-29T11:17:28.980 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment tear-down 2019-11-29T11:17:28.980 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] 4 tests from 1 test suite ran. (17926 ms total) 2019-11-29T11:17:28.980 INFO:tasks.workunit.client.0.smithi069.stdout:[ PASSED ] 4 tests. 2019-11-29T11:17:28.982 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-29T11:17:28.982 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-29T11:17:29.101 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-11-29T11:17:29.101 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-11-29T11:17:29.101 INFO:teuthology.orchestra.run.smithi069:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=656c8e8049c2c1acd363143c842c2edf1fe09b64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-11-29T11:17:29.158 INFO:tasks.workunit.client.0.smithi069.stdout:Running main() from gmock_main.cc 2019-11-29T11:17:29.158 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] Running 5 tests from 1 test suite. 2019-11-29T11:17:29.158 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment set-up. 2019-11-29T11:17:29.158 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 5 tests from cls_rgw 2019-11-29T11:17:29.159 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.test_implicit 2019-11-29T11:17:32.471 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.test_implicit (3314 ms) 2019-11-29T11:17:32.471 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-11-29T11:17:36.026 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3556 ms) 2019-11-29T11:17:36.027 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.test_put_snap 2019-11-29T11:17:44.865 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T11:17:44.855+0000 3aa7e700 -1 osd.2 pg_epoch: 105 pg[25.6( v 104'4 (0'0,104'4] local-lis/les=101/102 n=2 ec=101/101 lis/c=101/101 les/c/f=102/102/0 sis=101) [2,1] r=0 lpr=101 crt=103'3 lcod 103'3 mlcod 103'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-11-29T11:17:45.152 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.test_put_snap (9125 ms) 2019-11-29T11:17:45.152 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.test_explicit 2019-11-29T11:17:48.605 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.test_explicit (3455 ms) 2019-11-29T11:17:48.605 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.set 2019-11-29T11:17:52.082 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.set (3476 ms) 2019-11-29T11:17:52.082 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 5 tests from cls_rgw (22926 ms total) 2019-11-29T11:17:52.082 INFO:tasks.workunit.client.0.smithi069.stdout: 2019-11-29T11:17:52.082 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment tear-down 2019-11-29T11:17:52.082 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] 5 tests from 1 test suite ran. (22926 ms total) 2019-11-29T11:17:52.083 INFO:tasks.workunit.client.0.smithi069.stdout:[ PASSED ] 5 tests. 2019-11-29T11:17:52.085 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-29T11:17:52.085 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-29T11:17:52.153 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-11-29T11:17:52.153 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-11-29T11:17:52.154 INFO:teuthology.orchestra.run.smithi069:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=656c8e8049c2c1acd363143c842c2edf1fe09b64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-11-29T11:17:52.218 INFO:tasks.workunit.client.0.smithi069.stdout:Running main() from gmock_main.cc 2019-11-29T11:17:52.218 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] Running 13 tests from 1 test suite. 2019-11-29T11:17:52.218 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment set-up. 2019-11-29T11:17:52.219 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 13 tests from cls_rgw 2019-11-29T11:17:54.578 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.index_basic 2019-11-29T11:17:56.828 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.index_basic (2250 ms) 2019-11-29T11:17:56.829 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-11-29T11:17:59.139 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (2312 ms) 2019-11-29T11:17:59.140 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.index_remove_object 2019-11-29T11:18:01.640 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.index_remove_object (2500 ms) 2019-11-29T11:18:01.640 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.index_suggest 2019-11-29T11:18:39.044 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.index_suggest (37408 ms) 2019-11-29T11:18:39.044 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.index_list 2019-11-29T11:18:40.429 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.index_list (1385 ms) 2019-11-29T11:18:40.429 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.bi_list 2019-11-29T11:18:45.289 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.bi_list (4860 ms) 2019-11-29T11:18:45.289 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.gc_set 2019-11-29T11:18:46.033 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.gc_set (741 ms) 2019-11-29T11:18:46.033 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.gc_list 2019-11-29T11:18:46.668 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.gc_list (639 ms) 2019-11-29T11:18:46.669 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.gc_defer 2019-11-29T11:18:54.850 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.gc_defer (8182 ms) 2019-11-29T11:18:54.850 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.usage_basic 2019-11-29T11:19:05.731 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.usage_basic (10882 ms) 2019-11-29T11:19:05.731 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-11-29T11:19:05.784 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.usage_clear_no_obj (53 ms) 2019-11-29T11:19:05.784 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.usage_clear 2019-11-29T11:19:13.416 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.usage_clear (7633 ms) 2019-11-29T11:19:13.417 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-11-29T11:19:16.212 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] cls_rgw.bi_log_trim (2796 ms) 2019-11-29T11:19:16.378 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 13 tests from cls_rgw (81641 ms total) 2019-11-29T11:19:16.378 INFO:tasks.workunit.client.0.smithi069.stdout: 2019-11-29T11:19:16.378 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment tear-down 2019-11-29T11:19:16.379 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] 13 tests from 1 test suite ran. (84171 ms total) 2019-11-29T11:19:16.379 INFO:tasks.workunit.client.0.smithi069.stdout:[ PASSED ] 13 tests. 2019-11-29T11:19:16.381 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-29T11:19:16.381 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-29T11:19:16.503 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-11-29T11:19:16.504 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-11-29T11:19:16.504 INFO:teuthology.orchestra.run.smithi069:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=656c8e8049c2c1acd363143c842c2edf1fe09b64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-11-29T11:19:16.612 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] Running 6 tests from 1 test suite. 2019-11-29T11:19:16.612 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment set-up. 2019-11-29T11:19:18.532 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 6 tests from rgw_gc_log 2019-11-29T11:19:18.532 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-11-29T11:19:19.107 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.init_existing_queue (576 ms) 2019-11-29T11:19:19.108 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-11-29T11:19:19.238 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.init_existing_omap (130 ms) 2019-11-29T11:19:19.238 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-11-29T11:19:19.343 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (104 ms) 2019-11-29T11:19:19.343 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-11-29T11:19:19.396 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (54 ms) 2019-11-29T11:19:19.396 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-11-29T11:19:19.483 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.defer1_after_init (87 ms) 2019-11-29T11:19:19.483 INFO:tasks.workunit.client.0.smithi069.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-11-29T11:19:19.517 INFO:tasks.workunit.client.0.smithi069.stdout:[ OK ] rgw_gc_log.defer2_before_init (34 ms) 2019-11-29T11:19:19.517 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] 6 tests from rgw_gc_log (985 ms total) 2019-11-29T11:19:19.517 INFO:tasks.workunit.client.0.smithi069.stdout: 2019-11-29T11:19:19.517 INFO:tasks.workunit.client.0.smithi069.stdout:[----------] Global test environment tear-down 2019-11-29T11:19:19.823 INFO:tasks.workunit.client.0.smithi069.stdout:[==========] 6 tests from 1 test suite ran. (3213 ms total) 2019-11-29T11:19:19.823 INFO:tasks.workunit.client.0.smithi069.stdout:[ PASSED ] 6 tests. 2019-11-29T11:19:19.825 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-29T11:19:19.826 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-29T11:19:19.898 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-11-29T11:19:19.898 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-11-29T11:19:20.134 DEBUG:teuthology.parallel:result is None 2019-11-29T11:19:20.134 INFO:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-11-29T11:19:20.211 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-11-29T11:19:20.211 INFO:teuthology.orchestra.run.smithi069:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-11-29T11:19:20.284 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-11-29T11:19:20.284 INFO:teuthology.run_tasks:Running task ragweed... 2019-11-29T11:19:20.295 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-11-29T11:19:20.295 INFO:tasks.ragweed:Downloading ragweed... 2019-11-29T11:19:20.296 INFO:tasks.ragweed:Using branch 'wip-yuval-master' for ragweed 2019-11-29T11:19:20.296 INFO:teuthology.orchestra.run.smithi069:> git clone -b wip-yuval-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-11-29T11:19:20.395 INFO:teuthology.orchestra.run.smithi069.stderr:fatal: Remote branch wip-yuval-master not found in upstream origin 2019-11-29T11:19:20.398 INFO:teuthology.orchestra.run.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-11-29T11:19:20.398 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-11-29T11:19:20.399 INFO:teuthology.orchestra.run.smithi069:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-11-29T11:19:20.495 INFO:teuthology.orchestra.run.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-11-29T11:19:20.602 INFO:tasks.ragweed:Creating rgw users... 2019-11-29T11:19:20.603 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-11-29T11:19:20.603 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key MDVJDNYRBRXNBRCINYMD --secret wpoB3LGHFKXp8nUNSu05ARfbIedmizJIUQDDULUgcGhTNIFJSmZNnQ== --email sysuser.client.0+test@test.test --system true 2019-11-29T11:19:20.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.765+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.784+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.784+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.792+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.792+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.796+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.796+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.804+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:20.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.957+0000 7fe52c60fec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:19:20.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.957+0000 7fe52c60fec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:19:20.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.957+0000 7fe52c60fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:19:20.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.957+0000 7fe52c60fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:19:20.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.957+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.964+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:20.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.964+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.972+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:19:20.927 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.972+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.936 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.982+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:20.936 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.982+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.943 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.990+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:19:20.944 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.990+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.948 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.994+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.948 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.994+0000 7fe52c60fec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:19:20.948 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.994+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.953 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.999+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.953 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:20.999+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.960 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.006+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:20.960 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.006+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.017+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:19:20.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.017+0000 7fe52c60fec0 20 zonegroup default 2019-11-29T11:19:20.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.017+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.976 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.022+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.976 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.022+0000 7fe52c60fec0 10 Cannot find current period zone using local zone 2019-11-29T11:19:20.977 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.022+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.029+0000 7fe52c60fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.029+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:20.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.038+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:20.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.038+0000 7fe52c60fec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:21.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.057+0000 7fe52c60fec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:19:21.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.057+0000 7fe52c60fec0 20 zone default found 2019-11-29T11:19:21.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.057+0000 7fe52c60fec0 20 started sync module instance, tier type = 2019-11-29T11:19:21.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.057+0000 7fe52c60fec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:19:21.489 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=0 2019-11-29T11:19:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=1 2019-11-29T11:19:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=2 2019-11-29T11:19:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=3 2019-11-29T11:19:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=4 2019-11-29T11:19:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.536+0000 7fe52c60fec0 20 add_watcher() i=5 2019-11-29T11:19:21.533 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.579+0000 7fe52c60fec0 20 add_watcher() i=6 2019-11-29T11:19:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.591+0000 7fe52c60fec0 20 add_watcher() i=7 2019-11-29T11:19:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.591+0000 7fe52c60fec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:19:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.592+0000 7fe4ea7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:19:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.599+0000 7fe513fff700 0 -- 172.21.15.69:0/1049851045 >> v2:172.21.15.69:3300/0 conn(0x5589c25d4960 msgr2=0x5589c25d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:19:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.802+0000 7fe513fff700 0 --2- 172.21.15.69:0/1049851045 >> v2:172.21.15.69:3300/0 conn(0x5589c25d4960 0x5589c25d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-29T11:19:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:21.875+0000 7fe51983c700 0 -- 172.21.15.69:0/1150069451 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7fe45c017590 msgr2=0x7fe45c019a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:19:23.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.184+0000 7fe460ff9700 20 reqs_thread_entry: start 2019-11-29T11:19:23.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.184+0000 7fe45a7fc700 20 reqs_thread_entry: start 2019-11-29T11:19:23.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.184+0000 7fe52c60fec0 20 init_complete bucket index max shards: 0 2019-11-29T11:19:23.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.185+0000 7fe458ff9700 20 reqs_thread_entry: start 2019-11-29T11:19:23.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.185+0000 7fe52c60fec0 20 get_system_obj_state: rctx=0x7ffe5b993638 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x5589c2652650 s->prefetch_data=0 2019-11-29T11:19:23.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.185+0000 7fe52c60fec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-11-29T11:19:23.161 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.209+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-11-29T11:19:23.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.209+0000 7fe52c60fec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-29T11:19:23.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.209+0000 7fe52c60fec0 20 get_system_obj_state: rctx=0x7ffe5b993648 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x5589c26543e0 s->prefetch_data=0 2019-11-29T11:19:23.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.209+0000 7fe52c60fec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-11-29T11:19:23.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.235+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-11-29T11:19:23.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.235+0000 7fe52c60fec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-29T11:19:23.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.235+0000 7fe52c60fec0 20 get_system_obj_state: rctx=0x7ffe5b993648 obj=default.rgw.meta:users.keys:MDVJDNYRBRXNBRCINYMD state=0x5589c26543e0 s->prefetch_data=0 2019-11-29T11:19:23.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.235+0000 7fe52c60fec0 10 cache get: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD : miss 2019-11-29T11:19:23.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.275+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD info.flags=0x0 2019-11-29T11:19:23.229 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.275+0000 7fe52c60fec0 10 adding default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD to cache LRU end 2019-11-29T11:19:23.229 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.275+0000 7fe52c60fec0 20 get_system_obj_state: rctx=0x7ffe5b993258 obj=default.rgw.meta:users.keys:MDVJDNYRBRXNBRCINYMD state=0x5589c26543e0 s->prefetch_data=0 2019-11-29T11:19:23.229 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.275+0000 7fe52c60fec0 10 cache get: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD : type miss (requested=0x6, cached=0x0) 2019-11-29T11:19:23.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.300+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD info.flags=0x0 2019-11-29T11:19:23.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.300+0000 7fe52c60fec0 10 moving default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD to cache LRU end 2019-11-29T11:19:23.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.300+0000 7fe52c60fec0 20 get_system_obj_state: rctx=0x7ffe5b9933d8 obj=default.rgw.meta:users.keys:MDVJDNYRBRXNBRCINYMD state=0x5589c26543e0 s->prefetch_data=0 2019-11-29T11:19:23.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.300+0000 7fe52c60fec0 10 cache get: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD : type miss (requested=0x6, cached=0x0) 2019-11-29T11:19:23.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.309+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD info.flags=0x0 2019-11-29T11:19:23.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.309+0000 7fe52c60fec0 10 moving default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD to cache LRU end 2019-11-29T11:19:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.386+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-11-29T11:19:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.386+0000 7fe52c60fec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-29T11:19:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.386+0000 7fe52c60fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-11-29T11:19:23.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.433+0000 7fe4faffd700 10 RGWWatcher::handle_notify() notify_id 532575944704 cookie 94050159652016 notifier 4463 bl.length()=637 2019-11-29T11:19:23.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.433+0000 7fe4faffd700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-11-29T11:19:23.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.433+0000 7fe4faffd700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-29T11:19:23.645 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.691+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-11-29T11:19:23.645 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.691+0000 7fe52c60fec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-29T11:19:23.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.692+0000 7fe52c60fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-11-29T11:19:23.673 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.720+0000 7fe4faffd700 10 RGWWatcher::handle_notify() notify_id 532575944704 cookie 94050159661488 notifier 4463 bl.length()=202 2019-11-29T11:19:23.673 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.720+0000 7fe4faffd700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-11-29T11:19:23.673 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:23.720+0000 7fe4faffd700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-29T11:19:24.252 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.300+0000 7fe52c60fec0 10 cache put: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD info.flags=0x7 2019-11-29T11:19:24.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.300+0000 7fe52c60fec0 10 moving default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD to cache LRU end 2019-11-29T11:19:24.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.300+0000 7fe52c60fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=190 2019-11-29T11:19:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.313+0000 7fe4faffd700 10 RGWWatcher::handle_notify() notify_id 532575944705 cookie 94050159661488 notifier 4463 bl.length()=190 2019-11-29T11:19:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.313+0000 7fe4faffd700 10 cache put: name=default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD info.flags=0x7 2019-11-29T11:19:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.313+0000 7fe4faffd700 10 moving default.rgw.meta+users.keys+MDVJDNYRBRXNBRCINYMD to cache LRU end 2019-11-29T11:19:24.798 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T11:19:24.798 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "sysuser.client.0", 2019-11-29T11:19:24.798 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. sysuser.client.0", 2019-11-29T11:19:24.798 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "sysuser.client.0+test@test.test", 2019-11-29T11:19:24.798 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "sysuser.client.0", 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "MDVJDNYRBRXNBRCINYMD", 2019-11-29T11:19:24.799 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "wpoB3LGHFKXp8nUNSu05ARfbIedmizJIUQDDULUgcGhTNIFJSmZNnQ==" 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: "system": "true", 2019-11-29T11:19:24.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:19:24.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:19:24.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T11:19:24.803 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.898+0000 7fe52c60fec0 20 remove_watcher() i=0 2019-11-29T11:19:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.899+0000 7fe52c60fec0 2 removed watcher, disabling cache 2019-11-29T11:19:24.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.953+0000 7fe52c60fec0 20 remove_watcher() i=1 2019-11-29T11:19:24.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:24.997+0000 7fe52c60fec0 20 remove_watcher() i=2 2019-11-29T11:19:25.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.067+0000 7fe52c60fec0 20 remove_watcher() i=3 2019-11-29T11:19:25.088 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.135+0000 7fe52c60fec0 20 remove_watcher() i=4 2019-11-29T11:19:25.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.186+0000 7fe52c60fec0 20 remove_watcher() i=5 2019-11-29T11:19:25.199 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.247+0000 7fe52c60fec0 20 remove_watcher() i=6 2019-11-29T11:19:25.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.302+0000 7fe52c60fec0 20 remove_watcher() i=7 2019-11-29T11:19:25.262 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-11-29T11:19:25.262 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key ZYVPWRHDMVUCIRAQKQAJ --secret DbTaB/KRhZ6y+g3J4XdCim/6QfSYkLag8sw1V6z4WkXw5Bx8JISkcQ== --email ragweed.client.0+test@test.test --system false 2019-11-29T11:19:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.442+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.464+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.464+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.479+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.479+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.484+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.484+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.444 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.492+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:25.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.696+0000 7f2803834ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:19:25.649 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.696+0000 7f2803834ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:19:25.649 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.696+0000 7f2803834ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:19:25.649 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.696+0000 7f2803834ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:19:25.649 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.696+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.658 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.706+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:25.659 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.706+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.720+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:19:25.673 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.720+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.733+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.733+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.751+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:19:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.752+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.758+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.758+0000 7f2803834ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:19:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.758+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.764+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.764+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.775+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.775+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.798+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:19:25.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.798+0000 7f2803834ec0 20 zonegroup default 2019-11-29T11:19:25.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.798+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.807+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.807+0000 7f2803834ec0 10 Cannot find current period zone using local zone 2019-11-29T11:19:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.807+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.816+0000 7f2803834ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:19:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.816+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.825+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:19:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.825+0000 7f2803834ec0 20 rados->read ofs=0 len=0 2019-11-29T11:19:25.797 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.845+0000 7f2803834ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:19:25.797 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.845+0000 7f2803834ec0 20 zone default found 2019-11-29T11:19:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.845+0000 7f2803834ec0 20 started sync module instance, tier type = 2019-11-29T11:19:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:25.845+0000 7f2803834ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:19:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=0 2019-11-29T11:19:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=1 2019-11-29T11:19:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=2 2019-11-29T11:19:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=3 2019-11-29T11:19:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=4 2019-11-29T11:19:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.257+0000 7f2803834ec0 20 add_watcher() i=5 2019-11-29T11:19:26.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.284+0000 7f2803834ec0 20 add_watcher() i=6 2019-11-29T11:19:26.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.290+0000 7f2803834ec0 20 add_watcher() i=7 2019-11-29T11:19:26.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.290+0000 7f2803834ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:19:26.244 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.291+0000 7f27c17e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:19:26.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.294+0000 7f27f0260700 0 -- 172.21.15.69:0/1230875120 >> v2:172.21.15.69:3300/0 conn(0x55d0f75297e0 msgr2=0x55d0f752bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-29T11:19:26.294 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:26.342+0000 7f27f0a61700 0 -- 172.21.15.69:0/253917489 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x55d0f753aa50 msgr2=0x55d0f753ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:19:28.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.064+0000 7f27337fe700 20 reqs_thread_entry: start 2019-11-29T11:19:28.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.064+0000 7f27317fa700 20 reqs_thread_entry: start 2019-11-29T11:19:28.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.064+0000 7f2803834ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:19:28.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.065+0000 7f271f7fe700 20 reqs_thread_entry: start 2019-11-29T11:19:28.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.065+0000 7f2803834ec0 20 get_system_obj_state: rctx=0x7ffda737acb8 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55d0f75a7510 s->prefetch_data=0 2019-11-29T11:19:28.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.065+0000 7f2803834ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-11-29T11:19:28.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.074+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-11-29T11:19:28.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.074+0000 7f2803834ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-29T11:19:28.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.074+0000 7f2803834ec0 20 get_system_obj_state: rctx=0x7ffda737acc8 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x55d0f75a92a0 s->prefetch_data=0 2019-11-29T11:19:28.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.074+0000 7f2803834ec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-11-29T11:19:28.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.095+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-11-29T11:19:28.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.095+0000 7f2803834ec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-29T11:19:28.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.095+0000 7f2803834ec0 20 get_system_obj_state: rctx=0x7ffda737acc8 obj=default.rgw.meta:users.keys:ZYVPWRHDMVUCIRAQKQAJ state=0x55d0f75a92a0 s->prefetch_data=0 2019-11-29T11:19:28.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.095+0000 7f2803834ec0 10 cache get: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ : miss 2019-11-29T11:19:28.055 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.103+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ info.flags=0x0 2019-11-29T11:19:28.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.103+0000 7f2803834ec0 10 adding default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ to cache LRU end 2019-11-29T11:19:28.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.103+0000 7f2803834ec0 20 get_system_obj_state: rctx=0x7ffda737a8d8 obj=default.rgw.meta:users.keys:ZYVPWRHDMVUCIRAQKQAJ state=0x55d0f75a92a0 s->prefetch_data=0 2019-11-29T11:19:28.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.103+0000 7f2803834ec0 10 cache get: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ : type miss (requested=0x6, cached=0x0) 2019-11-29T11:19:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.109+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ info.flags=0x0 2019-11-29T11:19:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.109+0000 7f2803834ec0 10 moving default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ to cache LRU end 2019-11-29T11:19:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.109+0000 7f2803834ec0 20 get_system_obj_state: rctx=0x7ffda737aa58 obj=default.rgw.meta:users.keys:ZYVPWRHDMVUCIRAQKQAJ state=0x55d0f75a92a0 s->prefetch_data=0 2019-11-29T11:19:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.109+0000 7f2803834ec0 10 cache get: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ : type miss (requested=0x6, cached=0x0) 2019-11-29T11:19:28.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.115+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ info.flags=0x0 2019-11-29T11:19:28.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.115+0000 7f2803834ec0 10 moving default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ to cache LRU end 2019-11-29T11:19:28.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.166+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-11-29T11:19:28.119 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.166+0000 7f2803834ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-29T11:19:28.119 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.166+0000 7f2803834ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-11-29T11:19:28.129 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.176+0000 7f27e67fc700 10 RGWWatcher::handle_notify() notify_id 532575944706 cookie 94355990840560 notifier 4472 bl.length()=637 2019-11-29T11:19:28.129 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.176+0000 7f27e67fc700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-11-29T11:19:28.129 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.176+0000 7f27e67fc700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-29T11:19:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.470+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-11-29T11:19:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.470+0000 7f2803834ec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-29T11:19:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.470+0000 7f2803834ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-11-29T11:19:28.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.480+0000 7f27e67fc700 10 RGWWatcher::handle_notify() notify_id 532575944705 cookie 94355990821152 notifier 4472 bl.length()=202 2019-11-29T11:19:28.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.480+0000 7f27e67fc700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-11-29T11:19:28.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.480+0000 7f27e67fc700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-29T11:19:28.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.739+0000 7f2803834ec0 10 cache put: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ info.flags=0x7 2019-11-29T11:19:28.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.739+0000 7f2803834ec0 10 moving default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ to cache LRU end 2019-11-29T11:19:28.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.739+0000 7f2803834ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=190 2019-11-29T11:19:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.794+0000 7f27e67fc700 10 RGWWatcher::handle_notify() notify_id 532575944704 cookie 94355990858064 notifier 4472 bl.length()=190 2019-11-29T11:19:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.794+0000 7f27e67fc700 10 cache put: name=default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ info.flags=0x7 2019-11-29T11:19:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.794+0000 7f27e67fc700 10 moving default.rgw.meta+users.keys+ZYVPWRHDMVUCIRAQKQAJ to cache LRU end 2019-11-29T11:19:28.843 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "ragweed.client.0", 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. ragweed.client.0", 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "ragweed.client.0+test@test.test", 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T11:19:28.844 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "ragweed.client.0", 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "ZYVPWRHDMVUCIRAQKQAJ", 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "DbTaB/KRhZ6y+g3J4XdCim/6QfSYkLag8sw1V6z4WkXw5Bx8JISkcQ==" 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T11:19:28.845 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "system": "true", 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T11:19:28.846 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:19:28.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:19:28.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:19:28.849 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T11:19:28.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T11:19:28.849 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T11:19:28.849 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T11:19:28.849 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:28.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.933+0000 7f2803834ec0 20 remove_watcher() i=0 2019-11-29T11:19:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.933+0000 7f2803834ec0 2 removed watcher, disabling cache 2019-11-29T11:19:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:28.971+0000 7f2803834ec0 20 remove_watcher() i=1 2019-11-29T11:19:28.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.001+0000 7f2803834ec0 20 remove_watcher() i=2 2019-11-29T11:19:29.024 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.070+0000 7f2803834ec0 20 remove_watcher() i=3 2019-11-29T11:19:29.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.136+0000 7f2803834ec0 20 remove_watcher() i=4 2019-11-29T11:19:29.164 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.210+0000 7f2803834ec0 20 remove_watcher() i=5 2019-11-29T11:19:29.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.278+0000 7f2803834ec0 20 remove_watcher() i=6 2019-11-29T11:19:29.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:19:29.332+0000 7f2803834ec0 20 remove_watcher() i=7 2019-11-29T11:19:29.295 INFO:tasks.ragweed:Configuring ragweed... 2019-11-29T11:19:29.296 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-11-29T11:19:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-11-29T11:19:35.829 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T11:19:35.865 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T11:19:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T11:19:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T11:19:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T11:19:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T11:19:36.521 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T11:19:36.888 INFO:teuthology.orchestra.run.smithi069.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-11-29T11:19:37.749 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T11:19:37.750 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:19:37.750 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-11-29T11:19:37.752 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-11-29T11:19:37.754 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-11-29T11:19:37.791 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-11-29T11:19:37.791 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-11-29T11:19:37.791 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-11-29T11:19:37.792 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:19:37.792 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-11-29T11:19:37.793 INFO:teuthology.orchestra.run.smithi069.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-11-29T11:19:37.796 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-11-29T11:19:37.796 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T11:19:37.796 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-11-29T11:19:38.181 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T11:19:38.384 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:38.384 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T11:19:38.385 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:38.385 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-11-29T11:19:38.386 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout:Installing for dependencies: 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T11:19:38.387 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T11:19:38.388 INFO:teuthology.orchestra.run.smithi069.stdout:Install 4 Packages (+3 Dependent packages) 2019-11-29T11:19:38.388 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:38.388 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 1.7 M 2019-11-29T11:19:38.388 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 12 M 2019-11-29T11:19:38.389 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T11:19:38.666 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2019-11-29T11:19:38.666 INFO:teuthology.orchestra.run.smithi069.stdout:Total 6.0 MB/s | 1.7 MB 00:00 2019-11-29T11:19:38.673 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T11:19:38.688 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T11:19:38.746 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T11:19:38.746 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T11:19:39.427 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-11-29T11:19:39.788 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-11-29T11:19:40.066 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-11-29T11:19:40.724 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-11-29T11:19:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-11-29T11:19:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-11-29T11:19:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-11-29T11:19:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-11-29T11:19:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-11-29T11:19:42.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-11-29T11:19:42.344 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-11-29T11:19:42.443 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-11-29T11:19:42.511 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-11-29T11:19:42.973 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-11-29T11:19:43.023 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:43.023 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T11:19:43.023 INFO:teuthology.orchestra.run.smithi069.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-11-29T11:19:43.025 INFO:teuthology.orchestra.run.smithi069.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-11-29T11:19:43.025 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:43.025 INFO:teuthology.orchestra.run.smithi069.stdout:Dependency Installed: 2019-11-29T11:19:43.026 INFO:teuthology.orchestra.run.smithi069.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-11-29T11:19:43.026 INFO:teuthology.orchestra.run.smithi069.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-11-29T11:19:43.026 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:43.026 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T11:19:43.183 INFO:teuthology.orchestra.run.smithi069.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-11-29T11:19:44.639 INFO:teuthology.orchestra.run.smithi069.stdout:Installing setuptools, pip, wheel...done. 2019-11-29T11:19:45.175 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting pip 2019-11-29T11:19:45.264 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-11-29T11:19:45.511 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: pip 2019-11-29T11:19:45.511 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: pip 9.0.1 2019-11-29T11:19:45.580 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling pip-9.0.1: 2019-11-29T11:19:45.625 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled pip-9.0.1 2019-11-29T11:19:46.312 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed pip-19.3.1 2019-11-29T11:19:47.263 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T11:19:47.346 INFO:teuthology.orchestra.run.smithi069.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-11-29T11:19:48.234 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T11:19:48.479 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting nose 2019-11-29T11:19:48.525 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-11-29T11:19:48.651 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: nose 2019-11-29T11:19:48.725 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed nose-1.3.7 2019-11-29T11:19:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T11:19:49.876 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting setuptools==32.3.1 2019-11-29T11:19:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-11-29T11:19:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-11-29T11:19:50.044 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: setuptools 2019-11-29T11:19:50.044 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: setuptools 28.8.0 2019-11-29T11:19:50.057 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling setuptools-28.8.0: 2019-11-29T11:19:50.067 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-29T11:19:50.197 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed setuptools-32.3.1 2019-11-29T11:19:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T11:19:51.052 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting PyYAML 2019-11-29T11:19:51.091 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB) 2019-11-29T11:19:51.793 INFO:teuthology.orchestra.run.smithi069.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-11-29T11:19:51.943 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-11-29T11:19:51.943 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-11-29T11:19:52.303 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting bunch>=1.0.0 2019-11-29T11:19:52.315 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-11-29T11:19:52.972 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting gevent>=1.0 2019-11-29T11:19:52.987 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-11-29T11:19:53.471 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting httplib2 2019-11-29T11:19:53.480 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-11-29T11:19:54.880 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting lxml 2019-11-29T11:19:54.881 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-11-29T11:19:55.318 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-29T11:19:55.337 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T11:19:55.450 INFO:teuthology.orchestra.run.smithi069.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-11-29T11:19:55.450 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for PyYAML (setup.py): started 2019-11-29T11:19:56.152 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-11-29T11:19:56.152 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for PyYAML: filename=PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl size=46204 sha256=819865010d4ac360a36b2e2c0cfce953de51ad4d138d3d6f9da20a5915559bc7 2019-11-29T11:19:56.152 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030 2019-11-29T11:19:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for bunch (setup.py): started 2019-11-29T11:19:56.479 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-11-29T11:19:56.479 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=586c01a58c9a10913290221d58d14d4aabd790cb8f499a3b508b51c22649b5e0 2019-11-29T11:19:56.480 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-11-29T11:19:56.485 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for httplib2 (setup.py): started 2019-11-29T11:19:56.878 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-11-29T11:19:56.880 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=f94cc0bd166a33c5b719cb00971d6b0a8c8e0bcbf7c2ba5ee532c41225e0cae8 2019-11-29T11:19:56.880 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-11-29T11:19:56.889 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully built PyYAML bunch httplib2 2019-11-29T11:19:56.991 INFO:teuthology.orchestra.run.smithi069.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-11-29T11:19:56.991 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-11-29T11:19:56.992 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: PyYAML 3.10 2019-11-29T11:19:56.992 INFO:teuthology.orchestra.run.smithi069.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-11-29T11:19:56.992 INFO:teuthology.orchestra.run.smithi069.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-11-29T11:19:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: lxml 3.2.1 2019-11-29T11:19:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-11-29T11:19:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-11-29T11:19:58.072 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed PyYAML-5.1.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-11-29T11:19:58.431 INFO:teuthology.orchestra.run.smithi069.stdout:running develop 2019-11-29T11:19:58.495 INFO:teuthology.orchestra.run.smithi069.stdout:running egg_info 2019-11-29T11:19:58.495 INFO:teuthology.orchestra.run.smithi069.stdout:creating ragweed.egg-info 2019-11-29T11:19:58.495 INFO:teuthology.orchestra.run.smithi069.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-11-29T11:19:58.495 INFO:teuthology.orchestra.run.smithi069.stdout:writing ragweed.egg-info/PKG-INFO 2019-11-29T11:19:58.496 INFO:teuthology.orchestra.run.smithi069.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-11-29T11:19:58.496 INFO:teuthology.orchestra.run.smithi069.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-11-29T11:19:58.496 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-29T11:19:58.496 INFO:teuthology.orchestra.run.smithi069.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-29T11:19:58.498 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-29T11:19:58.498 INFO:teuthology.orchestra.run.smithi069.stdout:running build_ext 2019-11-29T11:19:58.499 INFO:teuthology.orchestra.run.smithi069.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-11-29T11:19:58.499 INFO:teuthology.orchestra.run.smithi069.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-11-29T11:19:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:58.499 INFO:teuthology.orchestra.run.smithi069.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-11-29T11:19:58.499 INFO:teuthology.orchestra.run.smithi069.stdout:Processing dependencies for ragweed==0.0.1 2019-11-29T11:19:58.521 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for isodate>=0.4.4 2019-11-29T11:19:58.521 INFO:teuthology.orchestra.run.smithi069.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-11-29T11:19:58.628 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-11-29T11:19:58.681 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: isodate 0.6.0 2019-11-29T11:19:58.681 INFO:teuthology.orchestra.run.smithi069.stdout:Processing isodate-0.6.0.tar.gz 2019-11-29T11:19:58.691 INFO:teuthology.orchestra.run.smithi069.stdout:Writing /tmp/easy_install-AknrFT/isodate-0.6.0/setup.cfg 2019-11-29T11:19:58.691 INFO:teuthology.orchestra.run.smithi069.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-AknrFT/isodate-0.6.0/egg-dist-tmp-0P_JCk 2019-11-29T11:19:59.003 INFO:teuthology.orchestra.run.smithi069.stderr:zip_safe flag not set; analyzing archive contents... 2019-11-29T11:19:59.018 INFO:teuthology.orchestra.run.smithi069.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.019 INFO:teuthology.orchestra.run.smithi069.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-11-29T11:19:59.021 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.021 INFO:teuthology.orchestra.run.smithi069.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-11-29T11:19:59.043 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for gevent==1.4.0 2019-11-29T11:19:59.043 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: gevent 1.4.0 2019-11-29T11:19:59.043 INFO:teuthology.orchestra.run.smithi069.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-11-29T11:19:59.045 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.045 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.045 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for bunch==1.0.1 2019-11-29T11:19:59.046 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: bunch 1.0.1 2019-11-29T11:19:59.046 INFO:teuthology.orchestra.run.smithi069.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-11-29T11:19:59.046 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.046 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.046 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for PyYAML==5.1.2 2019-11-29T11:19:59.047 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: PyYAML 5.1.2 2019-11-29T11:19:59.047 INFO:teuthology.orchestra.run.smithi069.stdout:Adding PyYAML 5.1.2 to easy-install.pth file 2019-11-29T11:19:59.047 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.047 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.048 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for boto==2.49.0 2019-11-29T11:19:59.049 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: boto 2.49.0 2019-11-29T11:19:59.049 INFO:teuthology.orchestra.run.smithi069.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-11-29T11:19:59.049 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.049 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.050 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for six==1.9.0 2019-11-29T11:19:59.050 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: six 1.9.0 2019-11-29T11:19:59.050 INFO:teuthology.orchestra.run.smithi069.stdout:Adding six 1.9.0 to easy-install.pth file 2019-11-29T11:19:59.050 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.050 INFO:teuthology.orchestra.run.smithi069.stdout:Using /usr/lib/python2.7/site-packages 2019-11-29T11:19:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for greenlet==0.4.15 2019-11-29T11:19:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: greenlet 0.4.15 2019-11-29T11:19:59.058 INFO:teuthology.orchestra.run.smithi069.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-11-29T11:19:59.058 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:19:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-29T11:19:59.059 INFO:teuthology.orchestra.run.smithi069.stdout:Finished processing dependencies for ragweed==0.0.1 2019-11-29T11:19:59.081 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-11-29T11:19:59.136 INFO:tasks.ragweed:Configuring boto... 2019-11-29T11:19:59.136 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/boto.cfg 2019-11-29T11:19:59.267 INFO:teuthology.orchestra.run.smithi069:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-11-29T11:20:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-11-29T11:21:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-11-29T11:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-11-29T11:21:50.570 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-11-29T11:21:54.185 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:09:06.984 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.229 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:09:07.017 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.233 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:09:07.048 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.269 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:09:07.078 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.315 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:09:07.104 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.323 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:09:07.137 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.920 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:09:07.735 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:54.996 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:09:07.804 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:21:55.060 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:09:07.847 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:22:18.832 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-11-29T11:23:36.189 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-11-29T11:24:00.968 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:11:13.797 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:00.982 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:11:13.811 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:00.988 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:11:13.817 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:00.997 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:11:13.789 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:00.998 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:11:13.826 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:01.002 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:11:13.794 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:01.114 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:11:13.942 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:01.138 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:11:13.967 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:01.181 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:11:13.973 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:24:26.613 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-11-29T11:26:11.529 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-11-29T11:26:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-11-29T11:26:47.950 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T11:26:47.950 INFO:teuthology.orchestra.run.smithi069.stderr:---------------------------------------------------------------------- 2019-11-29T11:26:47.950 INFO:teuthology.orchestra.run.smithi069.stderr:Ran 9 tests in 408.297s 2019-11-29T11:26:47.951 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T11:26:47.951 INFO:teuthology.orchestra.run.smithi069.stderr:OK 2019-11-29T11:26:47.978 INFO:tasks.ragweed:Cleaning up boto... 2019-11-29T11:26:47.978 INFO:teuthology.orchestra.run.smithi069:> rm /home/ubuntu/cephtest/boto.cfg 2019-11-29T11:26:48.042 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-11-29T11:26:48.200 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.293+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.324+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.324+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.335+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.335+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.340+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.340+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.350+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:48.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.543+0000 7f1a38780ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:26:48.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.543+0000 7f1a38780ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:26:48.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.543+0000 7f1a38780ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:26:48.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.543+0000 7f1a38780ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:26:48.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.543+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.549+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.549+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.464 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.558+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:26:48.465 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.558+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.473 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.567+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:48.473 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.567+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.578+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:26:48.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.578+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.582+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.582+0000 7f1a38780ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:26:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.582+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.496 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.590+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.496 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.590+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.598+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:48.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.599+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.606+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:26:48.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.606+0000 7f1a38780ec0 20 zonegroup default 2019-11-29T11:26:48.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.606+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.517 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.611+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.517 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.611+0000 7f1a38780ec0 10 Cannot find current period zone using local zone 2019-11-29T11:26:48.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.611+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.521 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.615+0000 7f1a38780ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:48.521 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.615+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.622+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:48.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.622+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.630+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:26:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.630+0000 7f1a38780ec0 20 zone default found 2019-11-29T11:26:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.630+0000 7f1a38780ec0 20 started sync module instance, tier type = 2019-11-29T11:26:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.630+0000 7f1a38780ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:26:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:48.924+0000 7f1a259ad700 0 -- 172.21.15.69:0/3765422287 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x5612027309a0 msgr2=0x561202732dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:26:49.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=0 2019-11-29T11:26:49.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=1 2019-11-29T11:26:49.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=2 2019-11-29T11:26:49.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=3 2019-11-29T11:26:49.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=4 2019-11-29T11:26:49.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=5 2019-11-29T11:26:49.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.099+0000 7f1a38780ec0 20 add_watcher() i=6 2019-11-29T11:26:49.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.194+0000 7f1a38780ec0 20 add_watcher() i=7 2019-11-29T11:26:49.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.194+0000 7f1a38780ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:26:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.194+0000 7f19f67dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:26:49.109 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.203+0000 7f1a259ad700 0 -- 172.21.15.69:0/2318759697 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56120274b4f0 msgr2=0x561202751d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:26:49.312 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:49.405+0000 7f1a259ad700 0 --2- 172.21.15.69:0/2318759697 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56120274b4f0 0x561202751d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-29T11:26:50.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.899+0000 7f196cff9700 20 reqs_thread_entry: start 2019-11-29T11:26:50.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.899+0000 7f19667fc700 20 reqs_thread_entry: start 2019-11-29T11:26:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.899+0000 7f1a38780ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:26:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.899+0000 7f1964ff9700 20 reqs_thread_entry: start 2019-11-29T11:26:50.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.900+0000 7f1a38780ec0 20 get_system_obj_state: rctx=0x7ffd0abfecd8 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x5612027d0110 s->prefetch_data=0 2019-11-29T11:26:50.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.900+0000 7f1a38780ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-11-29T11:26:50.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-11-29T11:26:50.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-29T11:26:50.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:50.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:50.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T11:26:50.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.917+0000 7f1a38780ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:50.837 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.931+0000 7f1a38780ec0 20 rados_obj.operate() r=0 bl.length=448 2019-11-29T11:26:50.837 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.931+0000 7f1a38780ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-11-29T11:26:50.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.931+0000 7f1a38780ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-29T11:26:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.938+0000 7f1a38780ec0 10 removing key index: MDVJDNYRBRXNBRCINYMD 2019-11-29T11:26:50.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.938+0000 7f1a38780ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=170 2019-11-29T11:26:50.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:50.946+0000 7f1a06ffd700 10 RGWWatcher::handle_notify() notify_id 571230650371 cookie 94635350543472 notifier 4487 bl.length()=170 2019-11-29T11:26:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.029+0000 7f1a38780ec0 10 removing email index: sysuser.client.0+test@test.test 2019-11-29T11:26:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.029+0000 7f1a38780ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-11-29T11:26:50.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.046+0000 7f1a06ffd700 10 RGWWatcher::handle_notify() notify_id 571230650372 cookie 94635350543472 notifier 4487 bl.length()=182 2019-11-29T11:26:51.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.231+0000 7f1a38780ec0 10 removing user buckets index 2019-11-29T11:26:51.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.231+0000 7f1a38780ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-11-29T11:26:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.289+0000 7f1a06ffd700 10 RGWWatcher::handle_notify() notify_id 571230650368 cookie 94635350526960 notifier 4487 bl.length()=173 2019-11-29T11:26:51.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.601+0000 7f1a38780ec0 10 removing user index: sysuser.client.0 2019-11-29T11:26:51.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.601+0000 7f1a38780ec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-11-29T11:26:51.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.601+0000 7f1a38780ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-11-29T11:26:51.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.615+0000 7f1a06ffd700 10 RGWWatcher::handle_notify() notify_id 571230650370 cookie 94635350525200 notifier 4487 bl.length()=165 2019-11-29T11:26:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.892+0000 7f1a38780ec0 20 remove_watcher() i=0 2019-11-29T11:26:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.892+0000 7f1a38780ec0 2 removed watcher, disabling cache 2019-11-29T11:26:51.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.941+0000 7f1a38780ec0 20 remove_watcher() i=1 2019-11-29T11:26:51.897 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:51.993+0000 7f1a38780ec0 20 remove_watcher() i=2 2019-11-29T11:26:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.054+0000 7f1a38780ec0 20 remove_watcher() i=3 2019-11-29T11:26:52.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.113+0000 7f1a38780ec0 20 remove_watcher() i=4 2019-11-29T11:26:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.164+0000 7f1a38780ec0 20 remove_watcher() i=5 2019-11-29T11:26:52.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.231+0000 7f1a38780ec0 20 remove_watcher() i=6 2019-11-29T11:26:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.297+0000 7f1a38780ec0 20 remove_watcher() i=7 2019-11-29T11:26:52.216 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-11-29T11:26:52.379 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.474+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.495+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.495+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.409 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.505+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.409 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.505+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.511+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.511+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.521+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.715+0000 7f6a122b6ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:26:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.715+0000 7f6a122b6ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:26:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.715+0000 7f6a122b6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:26:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.715+0000 7f6a122b6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:26:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.715+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.726+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.726+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.745+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:26:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.745+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.755+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:52.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.755+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.769+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:26:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.770+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.774+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.774+0000 7f6a122b6ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:26:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.775+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.779+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.779+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.789+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.789+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.808+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:26:52.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.808+0000 7f6a122b6ec0 20 zonegroup default 2019-11-29T11:26:52.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.808+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.813+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.813+0000 7f6a122b6ec0 10 Cannot find current period zone using local zone 2019-11-29T11:26:52.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.813+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.723 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.818+0000 7f6a122b6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:26:52.723 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.818+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.731 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.826+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:26:52.731 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.826+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.835+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:26:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.835+0000 7f6a122b6ec0 20 zone default found 2019-11-29T11:26:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.835+0000 7f6a122b6ec0 20 started sync module instance, tier type = 2019-11-29T11:26:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:52.836+0000 7f6a122b6ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:26:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=0 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=1 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=2 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=3 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=4 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=5 2019-11-29T11:26:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.243+0000 7f6a122b6ec0 20 add_watcher() i=6 2019-11-29T11:26:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.275+0000 7f6a122b6ec0 20 add_watcher() i=7 2019-11-29T11:26:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.275+0000 7f6a122b6ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:26:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.275+0000 7f69cefed700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:26:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.288+0000 7f69ffce4700 0 -- 172.21.15.69:0/2277956722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556288d0c250 msgr2=0x556288d0c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:26:53.220 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:53.315+0000 7f69ff4e3700 0 -- 172.21.15.69:0/1205797447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556288d16cd0 msgr2=0x556288d190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T11:26:54.701 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.796+0000 7f693d7fa700 20 reqs_thread_entry: start 2019-11-29T11:26:54.701 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.796+0000 7f6936ffd700 20 reqs_thread_entry: start 2019-11-29T11:26:54.701 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.796+0000 7f6a122b6ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:26:54.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.797+0000 7f69357fa700 20 reqs_thread_entry: start 2019-11-29T11:26:54.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.797+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc115450a8 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x556288d8aa70 s->prefetch_data=0 2019-11-29T11:26:54.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.797+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-11-29T11:26:54.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-11-29T11:26:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-29T11:26:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T11:26:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.814+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.828+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=448 2019-11-29T11:26:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.828+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-11-29T11:26:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.828+0000 7f6a122b6ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-29T11:26:54.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.876+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-conf state=0x556288d8c390 s->prefetch_data=0 2019-11-29T11:26:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.876+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-11-29T11:26:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-11-29T11:26:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-29T11:26:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:54.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-11-29T11:26:54.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.916+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.937+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=200 2019-11-29T11:26:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.937+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-11-29T11:26:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.938+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-29T11:26:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.938+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 state=0x556288d8e950 s->prefetch_data=0 2019-11-29T11:26:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.938+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 : miss 2019-11-29T11:26:54.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 info.flags=0x16 2019-11-29T11:26:54.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 to cache LRU end 2019-11-29T11:26:54.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:26:54.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.979+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:54.902 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=292 2019-11-29T11:26:54.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 info.flags=0x13 2019-11-29T11:26:54.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 to cache LRU end 2019-11-29T11:26:54.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:54.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:54.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:54.998+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:54.940 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.035+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-conf[42bb32be-1e05-491f-815b-a606bdf24d77.4355.1]) start [] num_entries 1100 2019-11-29T11:26:54.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.080+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-11-29T11:26:54.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-11-29T11:26:54.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-11-29T11:26:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.081+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:54.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.088+0000 7f6a122b6ec0 10 manifest: total_size = 3879 2019-11-29T11:26:54.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.088+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.13 2019-11-29T11:26:54.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.088+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:54.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.088+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:54.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.088+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:55.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.260+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.284+0000 7f6a122b6ec0 10 manifest: total_size = 5838 2019-11-29T11:26:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.284+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.30 2019-11-29T11:26:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.284+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.284+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.284+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.570+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.593+0000 7f6a122b6ec0 10 manifest: total_size = 9698 2019-11-29T11:26:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.593+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.56 2019-11-29T11:26:55.500 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.593+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.500 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.593+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:55.500 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.593+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.907+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.935+0000 7f6a122b6ec0 10 manifest: total_size = 3820 2019-11-29T11:26:55.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.935+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.72 2019-11-29T11:26:55.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.935+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:55.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.935+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:55.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:55.935+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:56.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.118+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.139+0000 7f6a122b6ec0 10 manifest: total_size = 3922 2019-11-29T11:26:56.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.139+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.87 2019-11-29T11:26:56.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.139+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.139+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:56.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.139+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:56.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.401+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.321 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.417+0000 7f6a122b6ec0 10 manifest: total_size = 12925 2019-11-29T11:26:56.321 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.417+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.120 2019-11-29T11:26:56.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.417+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.417+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:56.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.417+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:56.615 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.710+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.719+0000 7f6a122b6ec0 10 manifest: total_size = 4111 2019-11-29T11:26:56.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.719+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.190 2019-11-29T11:26:56.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.719+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.719+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:56.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.720+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.917+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.933+0000 7f6a122b6ec0 10 manifest: total_size = 3994 2019-11-29T11:26:56.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.933+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.247 2019-11-29T11:26:56.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.933+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:56.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.933+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:56.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:56.933+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:57.052 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.147+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:57.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.178+0000 7f6a122b6ec0 10 manifest: total_size = 6025 2019-11-29T11:26:57.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.178+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.323 2019-11-29T11:26:57.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.178+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x556288d93b78 s->prefetch_data=0 2019-11-29T11:26:57.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.178+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d93b78 2019-11-29T11:26:57.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.178+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 2019-11-29T11:26:57.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.377+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-conf[42bb32be-1e05-491f-815b-a606bdf24d77.4355.1]) start _multipart_[] num_entries 1001 2019-11-29T11:26:57.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.377+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:26:57.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.492+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[42bb32be-1e05-491f-815b-a606bdf24d77.4355.1])"; objs.size()=0; is_truncated=0 2019-11-29T11:26:57.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.639+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-conf state=0x556288d95240 s->prefetch_data=0 2019-11-29T11:26:57.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.639+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-11-29T11:26:57.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-11-29T11:26:57.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-29T11:26:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-11-29T11:26:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-11-29T11:26:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.659+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-11-29T11:26:57.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.719+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650368 cookie 93881690441216 notifier 4725 bl.length()=162 2019-11-29T11:26:57.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.899+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:42bb32be-1e05-491f-815b-a606bdf24d77.4355.1 from cache 2019-11-29T11:26:57.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.899+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=219 2019-11-29T11:26:57.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:57.909+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650369 cookie 93881690450176 notifier 4725 bl.length()=219 2019-11-29T11:26:58.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.476+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:26:58.382 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.476+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:26:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.491+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.505+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=229 2019-11-29T11:26:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.506+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-11-29T11:26:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.506+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-29T11:26:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.506+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:26:58.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.506+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 : miss 2019-11-29T11:26:58.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 info.flags=0x16 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 to cache LRU end 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:26:58.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:26:58.434 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.527+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=321 2019-11-29T11:26:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 info.flags=0x13 2019-11-29T11:26:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 to cache LRU end 2019-11-29T11:26:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.544+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 2019-11-29T11:26:58.496 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.591+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.2]) start [] num_entries 1100 2019-11-29T11:26:58.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.651+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-29T11:26:58.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.651+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x556288d95118 s->prefetch_data=0 2019-11-29T11:26:58.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.674+0000 7f6a122b6ec0 10 manifest: total_size = 18874368 2019-11-29T11:26:58.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.674+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.20 2019-11-29T11:26:58.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.674+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x556288d95118 s->prefetch_data=0 2019-11-29T11:26:58.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.674+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d95118 2019-11-29T11:26:58.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.674+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 2019-11-29T11:26:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:26:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:26:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:26:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:26:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:26:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2019-11-29T11:26:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:26:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.864+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2019-11-29T11:26:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.865+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.20 2019-11-29T11:26:58.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.958+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.2]) start _multipart_[] num_entries 1001 2019-11-29T11:26:58.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:58.958+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:26:58.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.007+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.2])"; objs.size()=0; is_truncated=0 2019-11-29T11:26:58.995 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.089+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:26:58.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.089+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:26:59.009 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.104+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-11-29T11:26:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.105+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-11-29T11:26:59.051 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.146+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650368 cookie 93881690444656 notifier 4725 bl.length()=191 2019-11-29T11:26:59.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.344+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2 from cache 2019-11-29T11:26:59.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.344+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=248 2019-11-29T11:26:59.259 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.354+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650370 cookie 93881690483776 notifier 4725 bl.length()=248 2019-11-29T11:26:59.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.793+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:26:59.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.793+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:26:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.812+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.827+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=236 2019-11-29T11:26:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.827+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-11-29T11:26:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.827+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-29T11:26:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.827+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 state=0x556288d932c0 s->prefetch_data=0 2019-11-29T11:26:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.827+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 : miss 2019-11-29T11:26:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 info.flags=0x16 2019-11-29T11:26:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 to cache LRU end 2019-11-29T11:26:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:26:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.846+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:26:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=328 2019-11-29T11:26:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 info.flags=0x13 2019-11-29T11:26:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 to cache LRU end 2019-11-29T11:26:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:26:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:26:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.878+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 2019-11-29T11:26:59.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:26:59.930+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.3]) start [] num_entries 1100 2019-11-29T11:26:59.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.018+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-29T11:26:59.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.018+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x556288d95118 s->prefetch_data=0 2019-11-29T11:26:59.940 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.035+0000 7f6a122b6ec0 10 manifest: total_size = 18874368 2019-11-29T11:26:59.940 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.035+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.40 2019-11-29T11:26:59.941 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.035+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x556288d95118 s->prefetch_data=0 2019-11-29T11:26:59.941 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.035+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d95118 2019-11-29T11:26:59.941 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.035+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:00.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:27:00.103 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2019-11-29T11:27:00.104 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:27:00.104 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2019-11-29T11:27:00.104 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.195+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.40 2019-11-29T11:27:00.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.258+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.3]) start _multipart_[] num_entries 1001 2019-11-29T11:27:00.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.258+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:00.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.320+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.3])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:00.355 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.451+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:27:00.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.451+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-11-29T11:27:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-29T11:27:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-11-29T11:27:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.475+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-11-29T11:27:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.492+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650371 cookie 93881690446528 notifier 4725 bl.length()=198 2019-11-29T11:27:00.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.863+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3 from cache 2019-11-29T11:27:00.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.863+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=255 2019-11-29T11:27:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:00.871+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650372 cookie 93881690468352 notifier 4725 bl.length()=255 2019-11-29T11:27:01.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.225+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:27:01.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.225+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-11-29T11:27:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-11-29T11:27:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-29T11:27:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:01.140 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:01.140 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.234+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.242+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=231 2019-11-29T11:27:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.242+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-11-29T11:27:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.242+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-29T11:27:01.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.242+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:01.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.242+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 : miss 2019-11-29T11:27:01.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 info.flags=0x16 2019-11-29T11:27:01.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 to cache LRU end 2019-11-29T11:27:01.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:01.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:01.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:01.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:01.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:01.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:01.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.263+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=323 2019-11-29T11:27:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 info.flags=0x13 2019-11-29T11:27:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 to cache LRU end 2019-11-29T11:27:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:01.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:01.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.284+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.324+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.4]) start [] num_entries 1100 2019-11-29T11:27:01.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.430+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo[NUMaZkuDPL.A.2FVoVWnURrNbzB9.ji] 2019-11-29T11:27:01.335 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.430+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.482+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x556288d93ae8 s->prefetch_data=0 2019-11-29T11:27:01.399 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.494+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 13t0sjroal92fus330igh8ah6admp4kl 2019-11-29T11:27:01.399 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.494+0000 7f6a122b6ec0 20 get_obj_state_impl: setting s->olh_tag to m3v4qfn140p8bj9dm4c195i2ojx92wbt 2019-11-29T11:27:01.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.495+0000 7f6a122b6ec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=m3v4qfn140p8bj9dm4c195i2ojx92wbt 2019-11-29T11:27:01.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.576+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.723+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.723+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.767+0000 7f6a122b6ec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-11-29T11:27:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.767+0000 7f6a122b6ec0 20 olh_log_entry: epoch=3 op=3 key=foo[NUMaZkuDPL.A.2FVoVWnURrNbzB9.ji] 2019-11-29T11:27:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.767+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-index-versioning.1:_:NUMaZkuDPL.A.2FVoVWnURrNbzB9.ji_foo state=0x556288da0fe8 s->prefetch_data=0 2019-11-29T11:27:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.786+0000 7f6a122b6ec0 10 manifest: total_size = 18874368 2019-11-29T11:27:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.786+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.55 2019-11-29T11:27:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.786+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-index-versioning.1:_:NUMaZkuDPL.A.2FVoVWnURrNbzB9.ji_foo state=0x556288da0fe8 s->prefetch_data=0 2019-11-29T11:27:01.691 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.786+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288da0fe8 2019-11-29T11:27:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.786+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2019-11-29T11:27:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2019-11-29T11:27:01.927 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2019-11-29T11:27:01.927 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:01.994+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.55 2019-11-29T11:27:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.115+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.115+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.281+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:02.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.396+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 2019-11-29T11:27:02.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.396+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.4]) start _multipart_[] num_entries 1001 2019-11-29T11:27:02.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.396+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:02.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.422+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.4])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:02.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.572+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:27:02.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.572+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:02.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-11-29T11:27:02.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-29T11:27:02.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:02.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:02.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:02.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-11-29T11:27:02.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.586+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-11-29T11:27:02.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.597+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650373 cookie 93881690446528 notifier 4725 bl.length()=193 2019-11-29T11:27:02.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.815+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4 from cache 2019-11-29T11:27:02.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.815+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=250 2019-11-29T11:27:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:02.829+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650371 cookie 93881690465952 notifier 4725 bl.length()=250 2019-11-29T11:27:03.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.149+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x556288d8e520 s->prefetch_data=0 2019-11-29T11:27:03.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.149+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-11-29T11:27:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-11-29T11:27:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-29T11:27:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.164+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.173+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-29T11:27:03.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.173+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-11-29T11:27:03.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.173+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-29T11:27:03.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.173+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:03.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.173+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 : miss 2019-11-29T11:27:03.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 info.flags=0x16 2019-11-29T11:27:03.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 to cache LRU end 2019-11-29T11:27:03.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.188+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-29T11:27:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 info.flags=0x13 2019-11-29T11:27:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 to cache LRU end 2019-11-29T11:27:03.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:03.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:03.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.206+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 2019-11-29T11:27:03.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.230+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.5]) start [] num_entries 1100 2019-11-29T11:27:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.265+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-29T11:27:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.265+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-simple.1:foo state=0x556288da6678 s->prefetch_data=0 2019-11-29T11:27:03.187 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.283+0000 7f6a122b6ec0 10 manifest: total_size = 1848 2019-11-29T11:27:03.187 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.283+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.71 2019-11-29T11:27:03.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.283+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-multipart-simple.1:foo state=0x556288da6678 s->prefetch_data=0 2019-11-29T11:27:03.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.283+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288da6678 2019-11-29T11:27:03.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.284+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 2019-11-29T11:27:03.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-29T11:27:03.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-11-29T11:27:03.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-11-29T11:27:03.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-11-29T11:27:03.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-11-29T11:27:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-11-29T11:27:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.463+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.71 2019-11-29T11:27:03.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.552+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.5]) start _multipart_[] num_entries 1001 2019-11-29T11:27:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.552+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.578+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.5])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:03.595 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.691+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:03.595 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.691+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:03.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-11-29T11:27:03.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-29T11:27:03.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:03.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:03.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:03.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-11-29T11:27:03.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.723+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-11-29T11:27:03.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.739+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650373 cookie 93881690451728 notifier 4725 bl.length()=183 2019-11-29T11:27:03.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.932+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5 from cache 2019-11-29T11:27:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.932+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=240 2019-11-29T11:27:03.848 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:03.945+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650369 cookie 93881690444656 notifier 4725 bl.length()=240 2019-11-29T11:27:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.344+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.344+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-11-29T11:27:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-11-29T11:27:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-29T11:27:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:04.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.365+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:04.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.375+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=227 2019-11-29T11:27:04.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.375+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-11-29T11:27:04.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.375+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-29T11:27:04.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.375+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:04.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.375+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 : miss 2019-11-29T11:27:04.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 info.flags=0x16 2019-11-29T11:27:04.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 to cache LRU end 2019-11-29T11:27:04.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:04.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:04.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:04.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:04.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:04.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:04.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.421+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:04.346 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=319 2019-11-29T11:27:04.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 info.flags=0x13 2019-11-29T11:27:04.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 to cache LRU end 2019-11-29T11:27:04.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:04.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:04.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.443+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:04.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.477+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.7]) start [] num_entries 1100 2019-11-29T11:27:04.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-29T11:27:04.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-11-29T11:27:04.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-11-29T11:27:04.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-29T11:27:04.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-11-29T11:27:04.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-11-29T11:27:04.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-11-29T11:27:04.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.526+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.538+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.538+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.108 2019-11-29T11:27:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.538+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.538+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:04.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.538+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:04.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.659+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.659+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.659+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.659+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.659+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.108 2019-11-29T11:27:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.728+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.736+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.736+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.109 2019-11-29T11:27:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.736+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.736+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.736+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.871+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.871+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.871+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.871+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.871+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.109 2019-11-29T11:27:04.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.960+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.871 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.968+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:04.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.968+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _JQ9Xud5dM7onFGBeHrL-GagcDHgu3Ac 2019-11-29T11:27:04.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.968+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:04.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.968+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:04.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:04.968+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:04.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.089+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.089+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.089+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.089+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.089+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _JQ9Xud5dM7onFGBeHrL-GagcDHgu3Ac 2019-11-29T11:27:05.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.165+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.173+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:05.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.173+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _RAsocJ2QmxC-Vy_6ZK9qxmQIDAwtT6R 2019-11-29T11:27:05.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.173+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.173+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:05.078 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.173+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:05.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.306+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.306+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.306+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _RAsocJ2QmxC-Vy_6ZK9qxmQIDAwtT6R 2019-11-29T11:27:05.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.350+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.363+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:05.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.363+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.112 2019-11-29T11:27:05.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.363+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.363+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:05.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.363+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.486+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.486+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.486+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.486+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.486+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.112 2019-11-29T11:27:05.466 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.563+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.581+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:05.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.581+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _yuFVdKMmo24cQvcOzY4tGOSZefk8Nt3 2019-11-29T11:27:05.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.581+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.581+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:05.486 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.582+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.786+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.786+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.786+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.786+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.786+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: _yuFVdKMmo24cQvcOzY4tGOSZefk8Nt3 2019-11-29T11:27:05.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.872+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.786 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.883+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:05.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.883+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.114 2019-11-29T11:27:05.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.883+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.883+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:05.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.883+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:05.891 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.987+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.891 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.987+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.891 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.987+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:05.891 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.987+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:05.892 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:05.988+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.114 2019-11-29T11:27:05.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.080+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.089+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:05.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.089+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _a7GzwedOpjgMs7yRlr-HQ30lBxwhYyH 2019-11-29T11:27:05.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.089+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x556288d01298 s->prefetch_data=0 2019-11-29T11:27:05.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.089+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d01298 2019-11-29T11:27:05.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.089+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.267+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.267+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.267+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _a7GzwedOpjgMs7yRlr-HQ30lBxwhYyH 2019-11-29T11:27:06.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.392+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.311 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.408+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:06.312 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.408+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.116 2019-11-29T11:27:06.312 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.408+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.312 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.408+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:06.312 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.408+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:06.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.581+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.581+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.581+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.116 2019-11-29T11:27:06.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.646+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.670+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:06.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.670+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _D3mNovQSeu7DrM6GxP7O_Rpe8X0RKDF 2019-11-29T11:27:06.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.670+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.670+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:06.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.670+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:06.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.815+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.815+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.815+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.815+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.815+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: _D3mNovQSeu7DrM6GxP7O_Rpe8X0RKDF 2019-11-29T11:27:06.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.895+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.907+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.907+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _kNpDVGEqgNV_oS6yCobqP1MeGNaiSYA 2019-11-29T11:27:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.907+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:06.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.907+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:06.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:06.907+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.035+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.035+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.035+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.035+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.035+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: _kNpDVGEqgNV_oS6yCobqP1MeGNaiSYA 2019-11-29T11:27:07.072 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.167+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:07.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.183+0000 7f6a122b6ec0 10 manifest: total_size = 5242880 2019-11-29T11:27:07.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.183+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.119 2019-11-29T11:27:07.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.183+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:07.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.183+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:07.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.184+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 2019-11-29T11:27:07.265 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.358+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:07.265 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.358+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-29T11:27:07.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.358+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.119 2019-11-29T11:27:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.471+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.7]) start _multipart_[] num_entries 1001 2019-11-29T11:27:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.471+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:07.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.501+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.7])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:07.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.588+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:07.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.588+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:07.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-11-29T11:27:07.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-29T11:27:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-11-29T11:27:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.606+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-11-29T11:27:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.621+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650374 cookie 93881690446528 notifier 4725 bl.length()=189 2019-11-29T11:27:07.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.795+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7 from cache 2019-11-29T11:27:07.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.795+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=246 2019-11-29T11:27:07.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:07.816+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650372 cookie 93881690465952 notifier 4725 bl.length()=246 2019-11-29T11:27:08.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.312+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:08.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.312+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-11-29T11:27:08.233 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-11-29T11:27:08.233 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-29T11:27:08.233 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:08.234 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:08.234 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:08.234 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.329+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:08.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.345+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=237 2019-11-29T11:27:08.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.345+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-11-29T11:27:08.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.345+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-29T11:27:08.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.345+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:08.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.345+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 : miss 2019-11-29T11:27:08.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 info.flags=0x16 2019-11-29T11:27:08.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 to cache LRU end 2019-11-29T11:27:08.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:08.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:08.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:08.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:08.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:08.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:08.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.374+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:08.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=329 2019-11-29T11:27:08.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 info.flags=0x13 2019-11-29T11:27:08.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 to cache LRU end 2019-11-29T11:27:08.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:08.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:08.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.400+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:08.361 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.457+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.9]) start [] num_entries 1100 2019-11-29T11:27:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-11-29T11:27:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-11-29T11:27:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-29T11:27:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-11-29T11:27:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-11-29T11:27:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-11-29T11:27:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.525+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.539+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.539+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.225 2019-11-29T11:27:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.539+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.539+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:08.444 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.539+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:08.601 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.697+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.225 2019-11-29T11:27:08.665 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.761+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.774+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.774+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.226 2019-11-29T11:27:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.774+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.774+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:08.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.774+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:08.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:08.940+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.226 2019-11-29T11:27:08.913 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.009+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.922 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.018+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:08.922 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.018+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _TUIiX-8YDsQR5BJAj632NEga4RF6cbo 2019-11-29T11:27:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.018+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.018+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.018+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:09.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.031 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.124+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _TUIiX-8YDsQR5BJAj632NEga4RF6cbo 2019-11-29T11:27:09.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.183+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.129 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.203+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:09.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.203+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _NMnAehu1Ia4Jl7ow7xskmNN6r11Bau5 2019-11-29T11:27:09.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.203+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.203+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:09.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.203+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.348+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.348+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.348+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.348+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.348+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: _NMnAehu1Ia4Jl7ow7xskmNN6r11Bau5 2019-11-29T11:27:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.500+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.509+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:09.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.509+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.234 2019-11-29T11:27:09.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.509+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.509+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:09.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.509+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:09.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:09.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.639+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.234 2019-11-29T11:27:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.744+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.656 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.753+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:09.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.753+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _LnPuUqH2G0PysXNb9LtNsk_ykldY0BB 2019-11-29T11:27:09.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.753+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.753+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:09.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.753+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:09.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:09.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-29T11:27:09.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.914+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _LnPuUqH2G0PysXNb9LtNsk_ykldY0BB 2019-11-29T11:27:09.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.959+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.990+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.990+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.236 2019-11-29T11:27:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.990+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.990+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:09.990+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:10.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:10.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:10.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:10.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:10.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:10.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:10.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:10.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:10.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.114+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.236 2019-11-29T11:27:10.072 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.168+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.182+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.183+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _h4xBYONgI1EnW9zf2RmD2Gw5oaCO173 2019-11-29T11:27:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.183+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.183+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.183+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.313+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.313+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.313+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.313+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-29T11:27:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.313+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: _h4xBYONgI1EnW9zf2RmD2Gw5oaCO173 2019-11-29T11:27:10.265 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.361+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.373+0000 7f6a122b6ec0 10 manifest: total_size = 45360 2019-11-29T11:27:10.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.373+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.243 2019-11-29T11:27:10.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.373+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.373+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:10.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.373+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:10.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-29T11:27:10.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-11-29T11:27:10.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=4134 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2019-11-29T11:27:10.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.481+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.243 2019-11-29T11:27:10.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.533+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.543+0000 7f6a122b6ec0 10 manifest: total_size = 45360 2019-11-29T11:27:10.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.543+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _20Hf_J8N7JM4ZX_GkkfKpMZPT3ce0r6 2019-11-29T11:27:10.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.543+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.543+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:10.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.543+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:10.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.661+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:10.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.661+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2019-11-29T11:27:10.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.661+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: _20Hf_J8N7JM4ZX_GkkfKpMZPT3ce0r6 2019-11-29T11:27:10.611 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.707+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.723+0000 7f6a122b6ec0 10 manifest: total_size = 45360 2019-11-29T11:27:10.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.723+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to _0QfXa_ct3wTKPgAOmQ8bByN2iGPmepM 2019-11-29T11:27:10.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.723+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.723+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:10.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.723+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:10.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.894+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:10.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.894+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2019-11-29T11:27:10.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.894+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _0QfXa_ct3wTKPgAOmQ8bByN2iGPmepM 2019-11-29T11:27:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.945+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.956+0000 7f6a122b6ec0 10 manifest: total_size = 45360 2019-11-29T11:27:10.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.956+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.246 2019-11-29T11:27:10.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.956+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:10.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.956+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:10.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:10.956+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-11-29T11:27:11.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2019-11-29T11:27:11.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2019-11-29T11:27:11.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=4134 2019-11-29T11:27:11.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2019-11-29T11:27:11.006 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.101+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.246 2019-11-29T11:27:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.149+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.9]) start _multipart_[] num_entries 1001 2019-11-29T11:27:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.149+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:11.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.185+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.9])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:11.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.322+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x556288d91c00 s->prefetch_data=0 2019-11-29T11:27:11.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.322+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:11.241 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-11-29T11:27:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.338+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-11-29T11:27:11.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.352+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650370 cookie 93881690450176 notifier 4725 bl.length()=199 2019-11-29T11:27:11.808 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.905+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9 from cache 2019-11-29T11:27:11.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.905+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=256 2019-11-29T11:27:11.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:11.928+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650369 cookie 93881690441216 notifier 4725 bl.length()=256 2019-11-29T11:27:12.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.887+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:12.791 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.887+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-11-29T11:27:12.808 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-11-29T11:27:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-29T11:27:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.905+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.921+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=232 2019-11-29T11:27:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.921+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-11-29T11:27:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.921+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-29T11:27:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.921+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.921+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 : miss 2019-11-29T11:27:12.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 info.flags=0x16 2019-11-29T11:27:12.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 to cache LRU end 2019-11-29T11:27:12.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:12.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:12.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:12.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:12.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:12.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:12.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.946+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:12.871 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=324 2019-11-29T11:27:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 info.flags=0x13 2019-11-29T11:27:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 to cache LRU end 2019-11-29T11:27:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.966+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 2019-11-29T11:27:12.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:12.999+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.8]) start [] num_entries 1100 2019-11-29T11:27:12.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.042+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-29T11:27:12.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.042+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-29T11:27:12.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.042+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-29T11:27:12.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.042+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.055+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.055+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.197 2019-11-29T11:27:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.055+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.055+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.055+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 2019-11-29T11:27:13.091 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.091 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.091 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.091 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.091 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:13.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:13.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:13.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:13.094 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.187+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.197 2019-11-29T11:27:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.258+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:13.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.266+0000 7f6a122b6ec0 10 manifest: total_size = 11534336 2019-11-29T11:27:13.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.266+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.204 2019-11-29T11:27:13.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.266+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:13.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.266+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:13.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.266+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-29T11:27:13.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-29T11:27:13.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:13.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-29T11:27:13.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-29T11:27:13.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-29T11:27:13.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.384+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.204 2019-11-29T11:27:13.337 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.434+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:13.348 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.444+0000 7f6a122b6ec0 10 manifest: total_size = 1140 2019-11-29T11:27:13.348 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.444+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.211 2019-11-29T11:27:13.348 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.444+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:13.348 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.444+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:13.349 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.444+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 2019-11-29T11:27:13.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-29T11:27:13.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-11-29T11:27:13.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-11-29T11:27:13.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-29T11:27:13.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-11-29T11:27:13.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-11-29T11:27:13.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-11-29T11:27:13.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=110 2019-11-29T11:27:13.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-11-29T11:27:13.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.605+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.211 2019-11-29T11:27:13.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.654+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.8]) start _multipart_[] num_entries 1001 2019-11-29T11:27:13.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.654+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.686+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.8])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:13.720 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.818+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x556288d94100 s->prefetch_data=0 2019-11-29T11:27:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.818+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:13.744 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-11-29T11:27:13.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-29T11:27:13.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:13.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:13.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:13.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-11-29T11:27:13.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.842+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-11-29T11:27:13.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:13.858+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650375 cookie 93881690468352 notifier 4725 bl.length()=194 2019-11-29T11:27:14.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.106+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8 from cache 2019-11-29T11:27:14.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.106+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=251 2019-11-29T11:27:14.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.120+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650370 cookie 93881690441216 notifier 4725 bl.length()=251 2019-11-29T11:27:14.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.879+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:14.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.879+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-11-29T11:27:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-11-29T11:27:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-29T11:27:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.900+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.912+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=222 2019-11-29T11:27:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.912+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-11-29T11:27:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.912+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-29T11:27:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.912+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.912+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 : miss 2019-11-29T11:27:14.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 info.flags=0x16 2019-11-29T11:27:14.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 to cache LRU end 2019-11-29T11:27:14.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:14.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.936+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:14.860 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=314 2019-11-29T11:27:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 info.flags=0x13 2019-11-29T11:27:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 to cache LRU end 2019-11-29T11:27:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:14.957+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 2019-11-29T11:27:14.929 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.026+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.6]) start [] num_entries 1100 2019-11-29T11:27:14.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.064+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-29T11:27:14.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.064+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-29T11:27:14.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.064+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-29T11:27:14.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.064+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:14.979 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.076+0000 7f6a122b6ec0 10 manifest: total_size = 10485760 2019-11-29T11:27:14.979 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.076+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.84 2019-11-29T11:27:14.979 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.076+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:14.979 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.076+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:14.980 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.076+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 2019-11-29T11:27:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.241+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.84 2019-11-29T11:27:15.177 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.275+0000 7f69cefed700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:27:15.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.324+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:15.236 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.334+0000 7f6a122b6ec0 10 manifest: total_size = 10485760 2019-11-29T11:27:15.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.334+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.85 2019-11-29T11:27:15.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.334+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:15.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.334+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:15.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.334+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 2019-11-29T11:27:15.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-29T11:27:15.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-11-29T11:27:15.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.499+0000 7f6a122b6ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 42bb32be-1e05-491f-815b-a606bdf24d77.4465.85 2019-11-29T11:27:15.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.568+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:15.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.580+0000 7f6a122b6ec0 10 manifest: total_size = 648332 2019-11-29T11:27:15.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.580+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.86 2019-11-29T11:27:15.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.580+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:15.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.580+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:15.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.580+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 2019-11-29T11:27:15.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.791+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.6]) start _multipart_[] num_entries 1001 2019-11-29T11:27:15.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.791+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:15.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:15.941+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.6])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:15.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.036+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:15.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.036+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-11-29T11:27:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-29T11:27:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-11-29T11:27:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.060+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-11-29T11:27:15.978 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.076+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650376 cookie 93881690446528 notifier 4725 bl.length()=184 2019-11-29T11:27:16.777 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.875+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6 from cache 2019-11-29T11:27:16.777 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.875+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=241 2019-11-29T11:27:16.795 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:16.893+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650377 cookie 93881690468352 notifier 4725 bl.length()=241 2019-11-29T11:27:17.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.223+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:17.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.223+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-11-29T11:27:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-11-29T11:27:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-29T11:27:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.254+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.270+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-29T11:27:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.270+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-11-29T11:27:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.270+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-29T11:27:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.270+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.270+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 : miss 2019-11-29T11:27:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 info.flags=0x16 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 to cache LRU end 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.290+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:17.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-29T11:27:17.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 info.flags=0x13 2019-11-29T11:27:17.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 to cache LRU end 2019-11-29T11:27:17.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:17.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:17.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.311+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 2019-11-29T11:27:17.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.344+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.10]) start [] num_entries 1100 2019-11-29T11:27:17.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.382+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-29T11:27:17.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.382+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-29T11:27:17.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.382+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-29T11:27:17.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.382+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-29T11:27:17.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.382+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.416+0000 7f6a122b6ec0 10 manifest: total_size = 1048576 2019-11-29T11:27:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.416+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.306 2019-11-29T11:27:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.416+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.416+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:17.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.416+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 2019-11-29T11:27:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.631+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.661+0000 7f6a122b6ec0 10 manifest: total_size = 4134 2019-11-29T11:27:17.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.661+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.308 2019-11-29T11:27:17.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.661+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.661+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:17.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.661+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 2019-11-29T11:27:17.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.908+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.929+0000 7f6a122b6ec0 10 manifest: total_size = 110 2019-11-29T11:27:17.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.929+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.307 2019-11-29T11:27:17.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.929+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:17.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.929+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:17.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:17.929+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 2019-11-29T11:27:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.169+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.182+0000 7f6a122b6ec0 10 manifest: total_size = 49187 2019-11-29T11:27:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.182+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.305 2019-11-29T11:27:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.182+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.1:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.182+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.182+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 2019-11-29T11:27:18.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.495+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.10]) start _multipart_[] num_entries 1001 2019-11-29T11:27:18.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.495+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.599+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[42bb32be-1e05-491f-815b-a606bdf24d77.4355.10])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:18.622 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.720+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x556288da6c10 s->prefetch_data=0 2019-11-29T11:27:18.622 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.720+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-11-29T11:27:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-29T11:27:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-11-29T11:27:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.744+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-29T11:27:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:18.760+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650370 cookie 93881690444656 notifier 4725 bl.length()=181 2019-11-29T11:27:19.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.174+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:42bb32be-1e05-491f-815b-a606bdf24d77.4355.10 from cache 2019-11-29T11:27:19.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.174+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=239 2019-11-29T11:27:19.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.197+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650371 cookie 93881690450176 notifier 4725 bl.length()=239 2019-11-29T11:27:19.633 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.730+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:19.633 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.730+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-11-29T11:27:19.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-11-29T11:27:19.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-29T11:27:19.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:19.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:19.644 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:19.644 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.740+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.752+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-29T11:27:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.752+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-11-29T11:27:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.752+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-29T11:27:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.752+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 state=0x556288d94960 s->prefetch_data=0 2019-11-29T11:27:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.752+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 : miss 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 info.flags=0x16 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 to cache LRU end 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:19.674 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:19.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:19.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:19.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.770+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:19.689 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-29T11:27:19.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 info.flags=0x13 2019-11-29T11:27:19.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 to cache LRU end 2019-11-29T11:27:19.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:19.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:19.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.787+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 2019-11-29T11:27:19.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.826+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[42bb32be-1e05-491f-815b-a606bdf24d77.4355.11]) start [] num_entries 1100 2019-11-29T11:27:19.779 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.877+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-29T11:27:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.877+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-29T11:27:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.877+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-29T11:27:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.877+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-29T11:27:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.877+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:19.789 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.886+0000 7f6a122b6ec0 10 manifest: total_size = 524288 2019-11-29T11:27:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.886+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.313 2019-11-29T11:27:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.886+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.886+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:19.886+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 2019-11-29T11:27:19.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.081+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:19.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.096+0000 7f6a122b6ec0 10 manifest: total_size = 2078 2019-11-29T11:27:19.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.096+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.315 2019-11-29T11:27:19.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.096+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:19.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.096+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:19.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.096+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 2019-11-29T11:27:20.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.352+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:20.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.369+0000 7f6a122b6ec0 10 manifest: total_size = 527 2019-11-29T11:27:20.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.369+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.314 2019-11-29T11:27:20.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.369+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:20.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.369+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:20.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.369+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 2019-11-29T11:27:20.559 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.657+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.671+0000 7f6a122b6ec0 10 manifest: total_size = 24595 2019-11-29T11:27:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.671+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.312 2019-11-29T11:27:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.671+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.2:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.671+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.671+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 2019-11-29T11:27:20.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.848+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[42bb32be-1e05-491f-815b-a606bdf24d77.4355.11]) start _multipart_[] num_entries 1001 2019-11-29T11:27:20.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.848+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:20.883 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:20.980+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[42bb32be-1e05-491f-815b-a606bdf24d77.4355.11])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:20.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.084+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:20.987 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.084+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:21.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-11-29T11:27:21.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.116+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-29T11:27:21.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.131+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650371 cookie 93881690444656 notifier 4725 bl.length()=181 2019-11-29T11:27:21.613 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.710+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:42bb32be-1e05-491f-815b-a606bdf24d77.4355.11 from cache 2019-11-29T11:27:21.613 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.710+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=239 2019-11-29T11:27:21.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:21.728+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650372 cookie 93881690450176 notifier 4725 bl.length()=239 2019-11-29T11:27:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.473+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x556288d944a0 s->prefetch_data=0 2019-11-29T11:27:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.473+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-11-29T11:27:22.399 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.496+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-11-29T11:27:22.399 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.497+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-29T11:27:22.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.497+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:22.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.497+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:22.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.497+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-11-29T11:27:22.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.497+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:22.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.523+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-29T11:27:22.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.523+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-11-29T11:27:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.523+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-29T11:27:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.523+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544e70 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 state=0x556288d00ad0 s->prefetch_data=0 2019-11-29T11:27:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.523+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 : miss 2019-11-29T11:27:22.445 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 info.flags=0x16 2019-11-29T11:27:22.445 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 to cache LRU end 2019-11-29T11:27:22.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:22.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:22.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:22.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 20 Read xattr: user.rgw.acl 2019-11-29T11:27:22.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 20 Read xattr: user.rgw.idtag 2019-11-29T11:27:22.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 : type miss (requested=0x13, cached=0x16) 2019-11-29T11:27:22.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.542+0000 7f6a122b6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:22.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-29T11:27:22.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 info.flags=0x13 2019-11-29T11:27:22.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 to cache LRU end 2019-11-29T11:27:22.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:22.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-29T11:27:22.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.563+0000 7f6a122b6ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 2019-11-29T11:27:22.504 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.600+0000 7f6a122b6ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[42bb32be-1e05-491f-815b-a606bdf24d77.4355.12]) start [] num_entries 1100 2019-11-29T11:27:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.664+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-29T11:27:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.664+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-29T11:27:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.664+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-29T11:27:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.664+0000 7f6a122b6ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-29T11:27:22.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.664+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.674+0000 7f6a122b6ec0 10 manifest: total_size = 0 2019-11-29T11:27:22.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.674+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.320 2019-11-29T11:27:22.577 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.674+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:__ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.577 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.674+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:22.577 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.674+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 2019-11-29T11:27:22.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.827+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.838+0000 7f6a122b6ec0 10 manifest: total_size = 0 2019-11-29T11:27:22.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.838+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.322 2019-11-29T11:27:22.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.838+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.838+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:22.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:22.838+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 2019-11-29T11:27:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.003+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.013+0000 7f6a122b6ec0 10 manifest: total_size = 0 2019-11-29T11:27:22.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.013+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.321 2019-11-29T11:27:22.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.013+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:___ state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:22.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.013+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:22.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.013+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 2019-11-29T11:27:23.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.196+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.206+0000 7f6a122b6ec0 10 manifest: total_size = 0 2019-11-29T11:27:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.206+0000 7f6a122b6ec0 20 get_obj_state: setting s->obj_tag to 42bb32be-1e05-491f-815b-a606bdf24d77.4465.319 2019-11-29T11:27:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.206+0000 7f6a122b6ec0 20 get_obj_state: rctx=0x7ffc11544e90 obj=test-client.0-r-test-small-obj-data.3:obj state=0x556288d98418 s->prefetch_data=0 2019-11-29T11:27:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.206+0000 7f6a122b6ec0 20 prepare_atomic_modification: state is not atomic. state=0x556288d98418 2019-11-29T11:27:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.206+0000 7f6a122b6ec0 20 bucket index object: default.rgw.buckets.index:.dir.42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 2019-11-29T11:27:23.350 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.448+0000 7f6a122b6ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[42bb32be-1e05-491f-815b-a606bdf24d77.4355.12]) start _multipart_[] num_entries 1001 2019-11-29T11:27:23.350 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.448+0000 7f6a122b6ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-29T11:27:23.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.577+0000 7f6a122b6ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[42bb32be-1e05-491f-815b-a606bdf24d77.4355.12])"; objs.size()=0; is_truncated=0 2019-11-29T11:27:23.629 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.726+0000 7f6a122b6ec0 20 get_system_obj_state: rctx=0x7ffc11544828 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x556288d91c00 s->prefetch_data=0 2019-11-29T11:27:23.629 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.726+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-11-29T11:27:23.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-11-29T11:27:23.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-29T11:27:23.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T11:27:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T11:27:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-11-29T11:27:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-11-29T11:27:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.737+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-29T11:27:23.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:23.745+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650372 cookie 93881690444656 notifier 4725 bl.length()=181 2019-11-29T11:27:24.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.101+0000 7f6a122b6ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:42bb32be-1e05-491f-815b-a606bdf24d77.4355.12 from cache 2019-11-29T11:27:24.004 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.101+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=239 2019-11-29T11:27:24.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.116+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650373 cookie 93881690450176 notifier 4725 bl.length()=239 2019-11-29T11:27:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.590+0000 7f6a122b6ec0 10 removing key index: ZYVPWRHDMVUCIRAQKQAJ 2019-11-29T11:27:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.590+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=170 2019-11-29T11:27:24.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.608+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650373 cookie 93881690483776 notifier 4725 bl.length()=170 2019-11-29T11:27:24.791 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.890+0000 7f6a122b6ec0 10 removing email index: ragweed.client.0+test@test.test 2019-11-29T11:27:24.791 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.890+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-11-29T11:27:24.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:24.900+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650378 cookie 93881690446528 notifier 4725 bl.length()=182 2019-11-29T11:27:24.930 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.029+0000 7f6a122b6ec0 10 removing user buckets index 2019-11-29T11:27:24.930 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.029+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-11-29T11:27:24.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.036+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650379 cookie 93881690468352 notifier 4725 bl.length()=173 2019-11-29T11:27:25.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.175+0000 7f6a122b6ec0 10 removing user index: ragweed.client.0 2019-11-29T11:27:25.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.175+0000 7f6a122b6ec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-11-29T11:27:25.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.175+0000 7f6a122b6ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-11-29T11:27:25.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:25.206+0000 7f69f4ff9700 10 RGWWatcher::handle_notify() notify_id 571230650374 cookie 93881690451728 notifier 4725 bl.length()=165 2019-11-29T11:27:25.927 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.026+0000 7f6a122b6ec0 20 remove_watcher() i=0 2019-11-29T11:27:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.027+0000 7f6a122b6ec0 2 removed watcher, disabling cache 2019-11-29T11:27:25.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.083+0000 7f6a122b6ec0 20 remove_watcher() i=1 2019-11-29T11:27:26.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.123+0000 7f6a122b6ec0 20 remove_watcher() i=2 2019-11-29T11:27:26.096 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.195+0000 7f6a122b6ec0 20 remove_watcher() i=3 2019-11-29T11:27:26.161 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.260+0000 7f6a122b6ec0 20 remove_watcher() i=4 2019-11-29T11:27:26.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.314+0000 7f6a122b6ec0 20 remove_watcher() i=5 2019-11-29T11:27:26.297 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.396+0000 7f6a122b6ec0 20 remove_watcher() i=6 2019-11-29T11:27:26.363 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:26.462+0000 7f6a122b6ec0 20 remove_watcher() i=7 2019-11-29T11:27:26.374 INFO:tasks.ragweed:Removing ragweed... 2019-11-29T11:27:26.374 INFO:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/ragweed 2019-11-29T11:27:26.477 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-11-29T11:27:26.502 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-11-29T11:27:26.502 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-11-29T11:27:26.503 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-11-29T11:27:26.503 INFO:teuthology.orchestra.run.smithi069:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-11-29T11:27:26.554 INFO:teuthology.orchestra.run.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-11-29T11:27:30.492 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-11-29T11:27:30.493 INFO:teuthology.orchestra.run.smithi069:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-11-29T11:27:32.786 INFO:teuthology.orchestra.run.smithi069.stdout: Gradle will be installed under /opt/gradle 2019-11-29T11:27:32.837 INFO:teuthology.orchestra.run.smithi069.stderr:--2019-11-29 11:27:32-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-11-29T11:27:32.847 INFO:teuthology.orchestra.run.smithi069.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.190.9, 104.18.191.9, 2606:4700::6812:bf09, ... 2019-11-29T11:27:32.857 INFO:teuthology.orchestra.run.smithi069.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.190.9|:443... connected. 2019-11-29T11:27:32.888 INFO:teuthology.orchestra.run.smithi069.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-11-29T11:27:32.888 INFO:teuthology.orchestra.run.smithi069.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-11-29T11:27:32.888 INFO:teuthology.orchestra.run.smithi069.stderr:--2019-11-29 11:27:32-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-11-29T11:27:32.889 INFO:teuthology.orchestra.run.smithi069.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.159.20, 104.17.160.20, 2606:4700::6811:a014, ... 2019-11-29T11:27:32.896 INFO:teuthology.orchestra.run.smithi069.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.159.20|:443... connected. 2019-11-29T11:27:32.934 INFO:teuthology.orchestra.run.smithi069.stderr:HTTP request sent, awaiting response... 200 OK 2019-11-29T11:27:32.935 INFO:teuthology.orchestra.run.smithi069.stderr:Length: 75323957 (72M) [application/zip] 2019-11-29T11:27:32.935 INFO:teuthology.orchestra.run.smithi069.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-11-29T11:27:32.935 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T11:27:32.945 INFO:teuthology.orchestra.run.smithi069.stderr: 0K .......... .......... .......... .......... .......... 0% 6.39M 11s 2019-11-29T11:27:32.945 INFO:teuthology.orchestra.run.smithi069.stderr: 50K .......... .......... .......... .......... .......... 0% 73.5M 6s 2019-11-29T11:27:32.953 INFO:teuthology.orchestra.run.smithi069.stderr: 100K .......... .......... .......... .......... .......... 0% 7.33M 7s 2019-11-29T11:27:32.953 INFO:teuthology.orchestra.run.smithi069.stderr: 150K .......... .......... .......... .......... .......... 0% 91.3M 6s 2019-11-29T11:27:32.953 INFO:teuthology.orchestra.run.smithi069.stderr: 200K .......... .......... .......... .......... .......... 0% 131M 5s 2019-11-29T11:27:34.475 INFO:teuthology.orchestra.run.smithi069.stderr: 250K .......... .......... .......... .......... .......... 0% 7.20M 6s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 300K .......... .......... .......... .......... .......... 0% 118M 5s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 350K .......... .......... .......... .......... .......... 0% 107M 4s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 400K .......... .......... .......... .......... .......... 0% 123M 4s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 450K .......... .......... .......... .......... .......... 0% 124M 4s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 500K .......... .......... .......... .......... .......... 0% 130M 3s 2019-11-29T11:27:34.476 INFO:teuthology.orchestra.run.smithi069.stderr: 550K .......... .......... .......... .......... .......... 0% 8.91M 4s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 600K .......... .......... .......... .......... .......... 0% 131M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 650K .......... .......... .......... .......... .......... 0% 127M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 700K .......... .......... .......... .......... .......... 1% 146M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 750K .......... .......... .......... .......... .......... 1% 111M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 800K .......... .......... .......... .......... .......... 1% 132M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 850K .......... .......... .......... .......... .......... 1% 121M 3s 2019-11-29T11:27:34.477 INFO:teuthology.orchestra.run.smithi069.stderr: 900K .......... .......... .......... .......... .......... 1% 119M 3s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 950K .......... .......... .......... .......... .......... 1% 123M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1000K .......... .......... .......... .......... .......... 1% 151M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1050K .......... .......... .......... .......... .......... 1% 128M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1100K .......... .......... .......... .......... .......... 1% 144M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1150K .......... .......... .......... .......... .......... 1% 98.8M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1200K .......... .......... .......... .......... .......... 1% 115M 2s 2019-11-29T11:27:34.478 INFO:teuthology.orchestra.run.smithi069.stderr: 1250K .......... .......... .......... .......... .......... 1% 164M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1300K .......... .......... .......... .......... .......... 1% 127M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1350K .......... .......... .......... .......... .......... 1% 110M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1400K .......... .......... .......... .......... .......... 1% 31.7M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1450K .......... .......... .......... .......... .......... 2% 135M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1500K .......... .......... .......... .......... .......... 2% 157M 2s 2019-11-29T11:27:34.479 INFO:teuthology.orchestra.run.smithi069.stderr: 1550K .......... .......... .......... .......... .......... 2% 139M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1600K .......... .......... .......... .......... .......... 2% 213M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1650K .......... .......... .......... .......... .......... 2% 146M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1700K .......... .......... .......... .......... .......... 2% 111M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1750K .......... .......... .......... .......... .......... 2% 127M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1800K .......... .......... .......... .......... .......... 2% 149M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1850K .......... .......... .......... .......... .......... 2% 165M 2s 2019-11-29T11:27:34.480 INFO:teuthology.orchestra.run.smithi069.stderr: 1900K .......... .......... .......... .......... .......... 2% 168M 2s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 1950K .......... .......... .......... .......... .......... 2% 156M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2000K .......... .......... .......... .......... .......... 2% 178M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2050K .......... .......... .......... .......... .......... 2% 174M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2100K .......... .......... .......... .......... .......... 2% 161M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2150K .......... .......... .......... .......... .......... 2% 146M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2200K .......... .......... .......... .......... .......... 3% 176M 1s 2019-11-29T11:27:34.481 INFO:teuthology.orchestra.run.smithi069.stderr: 2250K .......... .......... .......... .......... .......... 3% 168M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2300K .......... .......... .......... .......... .......... 3% 158M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2350K .......... .......... .......... .......... .......... 3% 133M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2400K .......... .......... .......... .......... .......... 3% 157M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2450K .......... .......... .......... .......... .......... 3% 169M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2500K .......... .......... .......... .......... .......... 3% 164M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2550K .......... .......... .......... .......... .......... 3% 140M 1s 2019-11-29T11:27:34.482 INFO:teuthology.orchestra.run.smithi069.stderr: 2600K .......... .......... .......... .......... .......... 3% 187M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2650K .......... .......... .......... .......... .......... 3% 157M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2700K .......... .......... .......... .......... .......... 3% 152M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2750K .......... .......... .......... .......... .......... 3% 136M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2800K .......... .......... .......... .......... .......... 3% 147M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2850K .......... .......... .......... .......... .......... 3% 183M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2900K .......... .......... .......... .......... .......... 4% 183M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 2950K .......... .......... .......... .......... .......... 4% 154M 1s 2019-11-29T11:27:34.483 INFO:teuthology.orchestra.run.smithi069.stderr: 3000K .......... .......... .......... .......... .......... 4% 184M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3050K .......... .......... .......... .......... .......... 4% 175M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3100K .......... .......... .......... .......... .......... 4% 185M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3150K .......... .......... .......... .......... .......... 4% 159M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3200K .......... .......... .......... .......... .......... 4% 187M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3250K .......... .......... .......... .......... .......... 4% 180M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3300K .......... .......... .......... .......... .......... 4% 169M 1s 2019-11-29T11:27:34.484 INFO:teuthology.orchestra.run.smithi069.stderr: 3350K .......... .......... .......... .......... .......... 4% 165M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3400K .......... .......... .......... .......... .......... 4% 181M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3450K .......... .......... .......... .......... .......... 4% 189M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3500K .......... .......... .......... .......... .......... 4% 188M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3550K .......... .......... .......... .......... .......... 4% 153M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3600K .......... .......... .......... .......... .......... 4% 183M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3650K .......... .......... .......... .......... .......... 5% 183M 1s 2019-11-29T11:27:34.485 INFO:teuthology.orchestra.run.smithi069.stderr: 3700K .......... .......... .......... .......... .......... 5% 183M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 3750K .......... .......... .......... .......... .......... 5% 149M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 3800K .......... .......... .......... .......... .......... 5% 188M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 3850K .......... .......... .......... .......... .......... 5% 178M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 3900K .......... .......... .......... .......... .......... 5% 181M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 3950K .......... .......... .......... .......... .......... 5% 138M 1s 2019-11-29T11:27:34.486 INFO:teuthology.orchestra.run.smithi069.stderr: 4000K .......... .......... .......... .......... .......... 5% 175M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4050K .......... .......... .......... .......... .......... 5% 172M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4100K .......... .......... .......... .......... .......... 5% 175M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4150K .......... .......... .......... .......... .......... 5% 152M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4200K .......... .......... .......... .......... .......... 5% 188M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4250K .......... .......... .......... .......... .......... 5% 182M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4300K .......... .......... .......... .......... .......... 5% 171M 1s 2019-11-29T11:27:34.487 INFO:teuthology.orchestra.run.smithi069.stderr: 4350K .......... .......... .......... .......... .......... 5% 162M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4400K .......... .......... .......... .......... .......... 6% 179M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4450K .......... .......... .......... .......... .......... 6% 180M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4500K .......... .......... .......... .......... .......... 6% 192M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4550K .......... .......... .......... .......... .......... 6% 164M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4600K .......... .......... .......... .......... .......... 6% 178M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4650K .......... .......... .......... .......... .......... 6% 191M 1s 2019-11-29T11:27:34.488 INFO:teuthology.orchestra.run.smithi069.stderr: 4700K .......... .......... .......... .......... .......... 6% 186M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 4750K .......... .......... .......... .......... .......... 6% 173M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 4800K .......... .......... .......... .......... .......... 6% 202M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 4850K .......... .......... .......... .......... .......... 6% 197M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 4900K .......... .......... .......... .......... .......... 6% 203M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 4950K .......... .......... .......... .......... .......... 6% 172M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 5000K .......... .......... .......... .......... .......... 6% 198M 1s 2019-11-29T11:27:34.489 INFO:teuthology.orchestra.run.smithi069.stderr: 5050K .......... .......... .......... .......... .......... 6% 200M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5100K .......... .......... .......... .......... .......... 7% 202M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5150K .......... .......... .......... .......... .......... 7% 174M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5200K .......... .......... .......... .......... .......... 7% 196M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5250K .......... .......... .......... .......... .......... 7% 191M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5300K .......... .......... .......... .......... .......... 7% 203M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5350K .......... .......... .......... .......... .......... 7% 155M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5400K .......... .......... .......... .......... .......... 7% 179M 1s 2019-11-29T11:27:34.490 INFO:teuthology.orchestra.run.smithi069.stderr: 5450K .......... .......... .......... .......... .......... 7% 203M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5500K .......... .......... .......... .......... .......... 7% 189M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5550K .......... .......... .......... .......... .......... 7% 156M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5600K .......... .......... .......... .......... .......... 7% 189M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5650K .......... .......... .......... .......... .......... 7% 181M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5700K .......... .......... .......... .......... .......... 7% 187M 1s 2019-11-29T11:27:34.491 INFO:teuthology.orchestra.run.smithi069.stderr: 5750K .......... .......... .......... .......... .......... 7% 164M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 5800K .......... .......... .......... .......... .......... 7% 166M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 5850K .......... .......... .......... .......... .......... 8% 152M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 5900K .......... .......... .......... .......... .......... 8% 176M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 5950K .......... .......... .......... .......... .......... 8% 150M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 6000K .......... .......... .......... .......... .......... 8% 187M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 6050K .......... .......... .......... .......... .......... 8% 207M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 6100K .......... .......... .......... .......... .......... 8% 188M 1s 2019-11-29T11:27:34.492 INFO:teuthology.orchestra.run.smithi069.stderr: 6150K .......... .......... .......... .......... .......... 8% 149M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6200K .......... .......... .......... .......... .......... 8% 181M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6250K .......... .......... .......... .......... .......... 8% 176M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6300K .......... .......... .......... .......... .......... 8% 163M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6350K .......... .......... .......... .......... .......... 8% 146M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6400K .......... .......... .......... .......... .......... 8% 117M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6450K .......... .......... .......... .......... .......... 8% 95.7M 1s 2019-11-29T11:27:34.493 INFO:teuthology.orchestra.run.smithi069.stderr: 6500K .......... .......... .......... .......... .......... 8% 98.0M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6550K .......... .......... .......... .......... .......... 8% 101M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6600K .......... .......... .......... .......... .......... 9% 131M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6650K .......... .......... .......... .......... .......... 9% 106M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6700K .......... .......... .......... .......... .......... 9% 117M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6750K .......... .......... .......... .......... .......... 9% 93.3M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6800K .......... .......... .......... .......... .......... 9% 112M 1s 2019-11-29T11:27:34.494 INFO:teuthology.orchestra.run.smithi069.stderr: 6850K .......... .......... .......... .......... .......... 9% 114M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 6900K .......... .......... .......... .......... .......... 9% 116M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 6950K .......... .......... .......... .......... .......... 9% 102M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 7000K .......... .......... .......... .......... .......... 9% 113M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 7050K .......... .......... .......... .......... .......... 9% 123M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 7100K .......... .......... .......... .......... .......... 9% 110M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 7150K .......... .......... .......... .......... .......... 9% 86.4M 1s 2019-11-29T11:27:34.495 INFO:teuthology.orchestra.run.smithi069.stderr: 7200K .......... .......... .......... .......... .......... 9% 99.4M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7250K .......... .......... .......... .......... .......... 9% 98.1M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7300K .......... .......... .......... .......... .......... 9% 99.7M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7350K .......... .......... .......... .......... .......... 10% 94.7M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7400K .......... .......... .......... .......... .......... 10% 103M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7450K .......... .......... .......... .......... .......... 10% 97.8M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7500K .......... .......... .......... .......... .......... 10% 100M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7550K .......... .......... .......... .......... .......... 10% 77.9M 1s 2019-11-29T11:27:34.496 INFO:teuthology.orchestra.run.smithi069.stderr: 7600K .......... .......... .......... .......... .......... 10% 102M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7650K .......... .......... .......... .......... .......... 10% 96.9M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7700K .......... .......... .......... .......... .......... 10% 98.7M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7750K .......... .......... .......... .......... .......... 10% 98.7M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7800K .......... .......... .......... .......... .......... 10% 103M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7850K .......... .......... .......... .......... .......... 10% 101M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7900K .......... .......... .......... .......... .......... 10% 104M 1s 2019-11-29T11:27:34.497 INFO:teuthology.orchestra.run.smithi069.stderr: 7950K .......... .......... .......... .......... .......... 10% 83.1M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8000K .......... .......... .......... .......... .......... 10% 105M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8050K .......... .......... .......... .......... .......... 11% 129M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8100K .......... .......... .......... .......... .......... 11% 126M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8150K .......... .......... .......... .......... .......... 11% 96.9M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8200K .......... .......... .......... .......... .......... 11% 110M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8250K .......... .......... .......... .......... .......... 11% 106M 1s 2019-11-29T11:27:34.498 INFO:teuthology.orchestra.run.smithi069.stderr: 8300K .......... .......... .......... .......... .......... 11% 175M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8350K .......... .......... .......... .......... .......... 11% 141M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8400K .......... .......... .......... .......... .......... 11% 119M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8450K .......... .......... .......... .......... .......... 11% 122M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8500K .......... .......... .......... .......... .......... 11% 105M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8550K .......... .......... .......... .......... .......... 11% 122M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8600K .......... .......... .......... .......... .......... 11% 140M 1s 2019-11-29T11:27:34.499 INFO:teuthology.orchestra.run.smithi069.stderr: 8650K .......... .......... .......... .......... .......... 11% 147M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8700K .......... .......... .......... .......... .......... 11% 136M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8750K .......... .......... .......... .......... .......... 11% 111M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8800K .......... .......... .......... .......... .......... 12% 130M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8850K .......... .......... .......... .......... .......... 12% 130M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8900K .......... .......... .......... .......... .......... 12% 132M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 8950K .......... .......... .......... .......... .......... 12% 116M 1s 2019-11-29T11:27:34.500 INFO:teuthology.orchestra.run.smithi069.stderr: 9000K .......... .......... .......... .......... .......... 12% 133M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9050K .......... .......... .......... .......... .......... 12% 125M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9100K .......... .......... .......... .......... .......... 12% 142M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9150K .......... .......... .......... .......... .......... 12% 104M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9200K .......... .......... .......... .......... .......... 12% 134M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9250K .......... .......... .......... .......... .......... 12% 130M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9300K .......... .......... .......... .......... .......... 12% 131M 1s 2019-11-29T11:27:34.501 INFO:teuthology.orchestra.run.smithi069.stderr: 9350K .......... .......... .......... .......... .......... 12% 119M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9400K .......... .......... .......... .......... .......... 12% 130M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9450K .......... .......... .......... .......... .......... 12% 135M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9500K .......... .......... .......... .......... .......... 12% 141M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9550K .......... .......... .......... .......... .......... 13% 116M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9600K .......... .......... .......... .......... .......... 13% 142M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9650K .......... .......... .......... .......... .......... 13% 141M 1s 2019-11-29T11:27:34.502 INFO:teuthology.orchestra.run.smithi069.stderr: 9700K .......... .......... .......... .......... .......... 13% 145M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 9750K .......... .......... .......... .......... .......... 13% 130M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 9800K .......... .......... .......... .......... .......... 13% 142M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 9850K .......... .......... .......... .......... .......... 13% 146M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 9900K .......... .......... .......... .......... .......... 13% 133M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 9950K .......... .......... .......... .......... .......... 13% 119M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 10000K .......... .......... .......... .......... .......... 13% 144M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 10050K .......... .......... .......... .......... .......... 13% 146M 1s 2019-11-29T11:27:34.503 INFO:teuthology.orchestra.run.smithi069.stderr: 10100K .......... .......... .......... .......... .......... 13% 144M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10150K .......... .......... .......... .......... .......... 13% 135M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10200K .......... .......... .......... .......... .......... 13% 143M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10250K .......... .......... .......... .......... .......... 14% 134M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10300K .......... .......... .......... .......... .......... 14% 146M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10350K .......... .......... .......... .......... .......... 14% 107M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10400K .......... .......... .......... .......... .......... 14% 149M 1s 2019-11-29T11:27:34.504 INFO:teuthology.orchestra.run.smithi069.stderr: 10450K .......... .......... .......... .......... .......... 14% 159M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10500K .......... .......... .......... .......... .......... 14% 158M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10550K .......... .......... .......... .......... .......... 14% 136M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10600K .......... .......... .......... .......... .......... 14% 145M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10650K .......... .......... .......... .......... .......... 14% 144M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10700K .......... .......... .......... .......... .......... 14% 163M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10750K .......... .......... .......... .......... .......... 14% 115M 1s 2019-11-29T11:27:34.505 INFO:teuthology.orchestra.run.smithi069.stderr: 10800K .......... .......... .......... .......... .......... 14% 133M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 10850K .......... .......... .......... .......... .......... 14% 181M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 10900K .......... .......... .......... .......... .......... 14% 227M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 10950K .......... .......... .......... .......... .......... 14% 207M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 11000K .......... .......... .......... .......... .......... 15% 202M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 11050K .......... .......... .......... .......... .......... 15% 229M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 11100K .......... .......... .......... .......... .......... 15% 168M 1s 2019-11-29T11:27:34.506 INFO:teuthology.orchestra.run.smithi069.stderr: 11150K .......... .......... .......... .......... .......... 15% 111M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11200K .......... .......... .......... .......... .......... 15% 163M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11250K .......... .......... .......... .......... .......... 15% 231M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11300K .......... .......... .......... .......... .......... 15% 219M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11350K .......... .......... .......... .......... .......... 15% 206M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11400K .......... .......... .......... .......... .......... 15% 235M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11450K .......... .......... .......... .......... .......... 15% 226M 1s 2019-11-29T11:27:34.507 INFO:teuthology.orchestra.run.smithi069.stderr: 11500K .......... .......... .......... .......... .......... 15% 244M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11550K .......... .......... .......... .......... .......... 15% 211M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11600K .......... .......... .......... .......... .......... 15% 254M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11650K .......... .......... .......... .......... .......... 15% 247M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11700K .......... .......... .......... .......... .......... 15% 264M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11750K .......... .......... .......... .......... .......... 16% 196M 1s 2019-11-29T11:27:34.508 INFO:teuthology.orchestra.run.smithi069.stderr: 11800K .......... .......... .......... .......... .......... 16% 244M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 11850K .......... .......... .......... .......... .......... 16% 192M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 11900K .......... .......... .......... .......... .......... 16% 179M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 11950K .......... .......... .......... .......... .......... 16% 151M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 12000K .......... .......... .......... .......... .......... 16% 171M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 12050K .......... .......... .......... .......... .......... 16% 166M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 12100K .......... .......... .......... .......... .......... 16% 153M 1s 2019-11-29T11:27:34.509 INFO:teuthology.orchestra.run.smithi069.stderr: 12150K .......... .......... .......... .......... .......... 16% 158M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12200K .......... .......... .......... .......... .......... 16% 153M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12250K .......... .......... .......... .......... .......... 16% 147M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12300K .......... .......... .......... .......... .......... 16% 159M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12350K .......... .......... .......... .......... .......... 16% 139M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12400K .......... .......... .......... .......... .......... 16% 214M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12450K .......... .......... .......... .......... .......... 16% 145M 1s 2019-11-29T11:27:34.510 INFO:teuthology.orchestra.run.smithi069.stderr: 12500K .......... .......... .......... .......... .......... 17% 145M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12550K .......... .......... .......... .......... .......... 17% 133M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12600K .......... .......... .......... .......... .......... 17% 236M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12650K .......... .......... .......... .......... .......... 17% 238M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12700K .......... .......... .......... .......... .......... 17% 239M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12750K .......... .......... .......... .......... .......... 17% 215M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12800K .......... .......... .......... .......... .......... 17% 261M 1s 2019-11-29T11:27:34.511 INFO:teuthology.orchestra.run.smithi069.stderr: 12850K .......... .......... .......... .......... .......... 17% 250M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 12900K .......... .......... .......... .......... .......... 17% 265M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 12950K .......... .......... .......... .......... .......... 17% 203M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 13000K .......... .......... .......... .......... .......... 17% 241M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 13050K .......... .......... .......... .......... .......... 17% 252M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 13100K .......... .......... .......... .......... .......... 17% 243M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 13150K .......... .......... .......... .......... .......... 17% 191M 1s 2019-11-29T11:27:34.512 INFO:teuthology.orchestra.run.smithi069.stderr: 13200K .......... .......... .......... .......... .......... 18% 255M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13250K .......... .......... .......... .......... .......... 18% 251M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13300K .......... .......... .......... .......... .......... 18% 257M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13350K .......... .......... .......... .......... .......... 18% 225M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13400K .......... .......... .......... .......... .......... 18% 248M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13450K .......... .......... .......... .......... .......... 18% 256M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13500K .......... .......... .......... .......... .......... 18% 265M 1s 2019-11-29T11:27:34.513 INFO:teuthology.orchestra.run.smithi069.stderr: 13550K .......... .......... .......... .......... .......... 18% 212M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13600K .......... .......... .......... .......... .......... 18% 249M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13650K .......... .......... .......... .......... .......... 18% 251M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13700K .......... .......... .......... .......... .......... 18% 259M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13750K .......... .......... .......... .......... .......... 18% 225M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13800K .......... .......... .......... .......... .......... 18% 279M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13850K .......... .......... .......... .......... .......... 18% 268M 1s 2019-11-29T11:27:34.514 INFO:teuthology.orchestra.run.smithi069.stderr: 13900K .......... .......... .......... .......... .......... 18% 254M 1s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 13950K .......... .......... .......... .......... .......... 19% 213M 1s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14000K .......... .......... .......... .......... .......... 19% 263M 1s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14050K .......... .......... .......... .......... .......... 19% 259M 1s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14100K .......... .......... .......... .......... .......... 19% 255M 0s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14150K .......... .......... .......... .......... .......... 19% 240M 0s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14200K .......... .......... .......... .......... .......... 19% 269M 0s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14250K .......... .......... .......... .......... .......... 19% 266M 0s 2019-11-29T11:27:34.515 INFO:teuthology.orchestra.run.smithi069.stderr: 14300K .......... .......... .......... .......... .......... 19% 274M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14350K .......... .......... .......... .......... .......... 19% 233M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14400K .......... .......... .......... .......... .......... 19% 282M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14450K .......... .......... .......... .......... .......... 19% 260M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14500K .......... .......... .......... .......... .......... 19% 263M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14550K .......... .......... .......... .......... .......... 19% 223M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14600K .......... .......... .......... .......... .......... 19% 247M 0s 2019-11-29T11:27:34.516 INFO:teuthology.orchestra.run.smithi069.stderr: 14650K .......... .......... .......... .......... .......... 19% 261M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14700K .......... .......... .......... .......... .......... 20% 222M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14750K .......... .......... .......... .......... .......... 20% 174M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14800K .......... .......... .......... .......... .......... 20% 235M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14850K .......... .......... .......... .......... .......... 20% 240M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14900K .......... .......... .......... .......... .......... 20% 218M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 14950K .......... .......... .......... .......... .......... 20% 209M 0s 2019-11-29T11:27:34.517 INFO:teuthology.orchestra.run.smithi069.stderr: 15000K .......... .......... .......... .......... .......... 20% 232M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15050K .......... .......... .......... .......... .......... 20% 260M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15100K .......... .......... .......... .......... .......... 20% 265M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15150K .......... .......... .......... .......... .......... 20% 228M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15200K .......... .......... .......... .......... .......... 20% 208M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15250K .......... .......... .......... .......... .......... 20% 195M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15300K .......... .......... .......... .......... .......... 20% 211M 0s 2019-11-29T11:27:34.518 INFO:teuthology.orchestra.run.smithi069.stderr: 15350K .......... .......... .......... .......... .......... 20% 183M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15400K .......... .......... .......... .......... .......... 21% 225M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15450K .......... .......... .......... .......... .......... 21% 254M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15500K .......... .......... .......... .......... .......... 21% 271M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15550K .......... .......... .......... .......... .......... 21% 219M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15600K .......... .......... .......... .......... .......... 21% 260M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15650K .......... .......... .......... .......... .......... 21% 278M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15700K .......... .......... .......... .......... .......... 21% 277M 0s 2019-11-29T11:27:34.519 INFO:teuthology.orchestra.run.smithi069.stderr: 15750K .......... .......... .......... .......... .......... 21% 230M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 15800K .......... .......... .......... .......... .......... 21% 258M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 15850K .......... .......... .......... .......... .......... 21% 247M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 15900K .......... .......... .......... .......... .......... 21% 200M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 15950K .......... .......... .......... .......... .......... 21% 199M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 16000K .......... .......... .......... .......... .......... 21% 228M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 16050K .......... .......... .......... .......... .......... 21% 236M 0s 2019-11-29T11:27:34.520 INFO:teuthology.orchestra.run.smithi069.stderr: 16100K .......... .......... .......... .......... .......... 21% 227M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16150K .......... .......... .......... .......... .......... 22% 197M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16200K .......... .......... .......... .......... .......... 22% 239M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16250K .......... .......... .......... .......... .......... 22% 243M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16300K .......... .......... .......... .......... .......... 22% 210M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16350K .......... .......... .......... .......... .......... 22% 187M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16400K .......... .......... .......... .......... .......... 22% 230M 0s 2019-11-29T11:27:34.521 INFO:teuthology.orchestra.run.smithi069.stderr: 16450K .......... .......... .......... .......... .......... 22% 230M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16500K .......... .......... .......... .......... .......... 22% 231M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16550K .......... .......... .......... .......... .......... 22% 188M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16600K .......... .......... .......... .......... .......... 22% 256M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16650K .......... .......... .......... .......... .......... 22% 231M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16700K .......... .......... .......... .......... .......... 22% 228M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16750K .......... .......... .......... .......... .......... 22% 190M 0s 2019-11-29T11:27:34.522 INFO:teuthology.orchestra.run.smithi069.stderr: 16800K .......... .......... .......... .......... .......... 22% 232M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 16850K .......... .......... .......... .......... .......... 22% 209M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 16900K .......... .......... .......... .......... .......... 23% 235M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 16950K .......... .......... .......... .......... .......... 23% 194M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 17000K .......... .......... .......... .......... .......... 23% 218M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 17050K .......... .......... .......... .......... .......... 23% 237M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 17100K .......... .......... .......... .......... .......... 23% 275M 0s 2019-11-29T11:27:34.523 INFO:teuthology.orchestra.run.smithi069.stderr: 17150K .......... .......... .......... .......... .......... 23% 240M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17200K .......... .......... .......... .......... .......... 23% 275M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17250K .......... .......... .......... .......... .......... 23% 263M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17300K .......... .......... .......... .......... .......... 23% 239M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17350K .......... .......... .......... .......... .......... 23% 200M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17400K .......... .......... .......... .......... .......... 23% 236M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17450K .......... .......... .......... .......... .......... 23% 245M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17500K .......... .......... .......... .......... .......... 23% 232M 0s 2019-11-29T11:27:34.524 INFO:teuthology.orchestra.run.smithi069.stderr: 17550K .......... .......... .......... .......... .......... 23% 184M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17600K .......... .......... .......... .......... .......... 23% 246M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17650K .......... .......... .......... .......... .......... 24% 230M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17700K .......... .......... .......... .......... .......... 24% 226M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17750K .......... .......... .......... .......... .......... 24% 189M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17800K .......... .......... .......... .......... .......... 24% 225M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17850K .......... .......... .......... .......... .......... 24% 241M 0s 2019-11-29T11:27:34.525 INFO:teuthology.orchestra.run.smithi069.stderr: 17900K .......... .......... .......... .......... .......... 24% 229M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 17950K .......... .......... .......... .......... .......... 24% 175M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18000K .......... .......... .......... .......... .......... 24% 195M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18050K .......... .......... .......... .......... .......... 24% 216M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18100K .......... .......... .......... .......... .......... 24% 228M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18150K .......... .......... .......... .......... .......... 24% 185M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18200K .......... .......... .......... .......... .......... 24% 215M 0s 2019-11-29T11:27:34.526 INFO:teuthology.orchestra.run.smithi069.stderr: 18250K .......... .......... .......... .......... .......... 24% 213M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18300K .......... .......... .......... .......... .......... 24% 202M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18350K .......... .......... .......... .......... .......... 25% 170M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18400K .......... .......... .......... .......... .......... 25% 159M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18450K .......... .......... .......... .......... .......... 25% 201M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18500K .......... .......... .......... .......... .......... 25% 193M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18550K .......... .......... .......... .......... .......... 25% 167M 0s 2019-11-29T11:27:34.527 INFO:teuthology.orchestra.run.smithi069.stderr: 18600K .......... .......... .......... .......... .......... 25% 199M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18650K .......... .......... .......... .......... .......... 25% 195M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18700K .......... .......... .......... .......... .......... 25% 197M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18750K .......... .......... .......... .......... .......... 25% 167M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18800K .......... .......... .......... .......... .......... 25% 199M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18850K .......... .......... .......... .......... .......... 25% 204M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18900K .......... .......... .......... .......... .......... 25% 188M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 18950K .......... .......... .......... .......... .......... 25% 180M 0s 2019-11-29T11:27:34.528 INFO:teuthology.orchestra.run.smithi069.stderr: 19000K .......... .......... .......... .......... .......... 25% 200M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19050K .......... .......... .......... .......... .......... 25% 201M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19100K .......... .......... .......... .......... .......... 26% 195M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19150K .......... .......... .......... .......... .......... 26% 176M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19200K .......... .......... .......... .......... .......... 26% 196M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19250K .......... .......... .......... .......... .......... 26% 201M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19300K .......... .......... .......... .......... .......... 26% 191M 0s 2019-11-29T11:27:34.529 INFO:teuthology.orchestra.run.smithi069.stderr: 19350K .......... .......... .......... .......... .......... 26% 164M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19400K .......... .......... .......... .......... .......... 26% 197M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19450K .......... .......... .......... .......... .......... 26% 190M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19500K .......... .......... .......... .......... .......... 26% 193M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19550K .......... .......... .......... .......... .......... 26% 157M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19600K .......... .......... .......... .......... .......... 26% 214M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19650K .......... .......... .......... .......... .......... 26% 195M 0s 2019-11-29T11:27:34.530 INFO:teuthology.orchestra.run.smithi069.stderr: 19700K .......... .......... .......... .......... .......... 26% 183M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 19750K .......... .......... .......... .......... .......... 26% 177M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 19800K .......... .......... .......... .......... .......... 26% 204M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 19850K .......... .......... .......... .......... .......... 27% 186M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 19900K .......... .......... .......... .......... .......... 27% 195M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 19950K .......... .......... .......... .......... .......... 27% 166M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 20000K .......... .......... .......... .......... .......... 27% 197M 0s 2019-11-29T11:27:34.531 INFO:teuthology.orchestra.run.smithi069.stderr: 20050K .......... .......... .......... .......... .......... 27% 192M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20100K .......... .......... .......... .......... .......... 27% 203M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20150K .......... .......... .......... .......... .......... 27% 160M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20200K .......... .......... .......... .......... .......... 27% 190M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20250K .......... .......... .......... .......... .......... 27% 189M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20300K .......... .......... .......... .......... .......... 27% 202M 0s 2019-11-29T11:27:34.532 INFO:teuthology.orchestra.run.smithi069.stderr: 20350K .......... .......... .......... .......... .......... 27% 172M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20400K .......... .......... .......... .......... .......... 27% 201M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20450K .......... .......... .......... .......... .......... 27% 181M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20500K .......... .......... .......... .......... .......... 27% 186M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20550K .......... .......... .......... .......... .......... 28% 186M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20600K .......... .......... .......... .......... .......... 28% 204M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20650K .......... .......... .......... .......... .......... 28% 189M 0s 2019-11-29T11:27:34.533 INFO:teuthology.orchestra.run.smithi069.stderr: 20700K .......... .......... .......... .......... .......... 28% 221M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 20750K .......... .......... .......... .......... .......... 28% 169M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 20800K .......... .......... .......... .......... .......... 28% 212M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 20850K .......... .......... .......... .......... .......... 28% 211M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 20900K .......... .......... .......... .......... .......... 28% 217M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 20950K .......... .......... .......... .......... .......... 28% 166M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 21000K .......... .......... .......... .......... .......... 28% 194M 0s 2019-11-29T11:27:34.534 INFO:teuthology.orchestra.run.smithi069.stderr: 21050K .......... .......... .......... .......... .......... 28% 197M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21100K .......... .......... .......... .......... .......... 28% 209M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21150K .......... .......... .......... .......... .......... 28% 162M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21200K .......... .......... .......... .......... .......... 28% 208M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21250K .......... .......... .......... .......... .......... 28% 195M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21300K .......... .......... .......... .......... .......... 29% 212M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21350K .......... .......... .......... .......... .......... 29% 182M 0s 2019-11-29T11:27:34.535 INFO:teuthology.orchestra.run.smithi069.stderr: 21400K .......... .......... .......... .......... .......... 29% 212M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21450K .......... .......... .......... .......... .......... 29% 196M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21500K .......... .......... .......... .......... .......... 29% 207M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21550K .......... .......... .......... .......... .......... 29% 174M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21600K .......... .......... .......... .......... .......... 29% 198M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21650K .......... .......... .......... .......... .......... 29% 213M 0s 2019-11-29T11:27:34.536 INFO:teuthology.orchestra.run.smithi069.stderr: 21700K .......... .......... .......... .......... .......... 29% 209M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 21750K .......... .......... .......... .......... .......... 29% 180M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 21800K .......... .......... .......... .......... .......... 29% 214M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 21850K .......... .......... .......... .......... .......... 29% 218M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 21900K .......... .......... .......... .......... .......... 29% 209M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 21950K .......... .......... .......... .......... .......... 29% 178M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 22000K .......... .......... .......... .......... .......... 29% 191M 0s 2019-11-29T11:27:34.537 INFO:teuthology.orchestra.run.smithi069.stderr: 22050K .......... .......... .......... .......... .......... 30% 215M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22100K .......... .......... .......... .......... .......... 30% 210M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22150K .......... .......... .......... .......... .......... 30% 187M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22200K .......... .......... .......... .......... .......... 30% 211M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22250K .......... .......... .......... .......... .......... 30% 186M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22300K .......... .......... .......... .......... .......... 30% 216M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22350K .......... .......... .......... .......... .......... 30% 190M 0s 2019-11-29T11:27:34.538 INFO:teuthology.orchestra.run.smithi069.stderr: 22400K .......... .......... .......... .......... .......... 30% 214M 0s 2019-11-29T11:27:34.539 INFO:teuthology.orchestra.run.smithi069.stderr: 22450K .......... .......... .......... .......... .......... 30% 206M 0s 2019-11-29T11:27:34.539 INFO:teuthology.orchestra.run.smithi069.stderr: 22500K .......... .......... .......... .......... .......... 30% 193M 0s 2019-11-29T11:27:34.539 INFO:teuthology.orchestra.run.smithi069.stderr: 22550K .......... .......... .......... .......... .......... 30% 187M 0s 2019-11-29T11:27:34.539 INFO:teuthology.orchestra.run.smithi069.stderr: 22600K .......... .......... .......... .......... .......... 30% 216M 0s 2019-11-29T11:27:34.539 INFO:teuthology.orchestra.run.smithi069.stderr: 22650K .......... .......... .......... .......... .......... 30% 210M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22700K .......... .......... .......... .......... .......... 30% 205M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22750K .......... .......... .......... .......... .......... 30% 182M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22800K .......... .......... .......... .......... .......... 31% 207M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22850K .......... .......... .......... .......... .......... 31% 194M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22900K .......... .......... .......... .......... .......... 31% 208M 0s 2019-11-29T11:27:34.540 INFO:teuthology.orchestra.run.smithi069.stderr: 22950K .......... .......... .......... .......... .......... 31% 174M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23000K .......... .......... .......... .......... .......... 31% 211M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23050K .......... .......... .......... .......... .......... 31% 230M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23100K .......... .......... .......... .......... .......... 31% 204M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23150K .......... .......... .......... .......... .......... 31% 171M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23200K .......... .......... .......... .......... .......... 31% 210M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23250K .......... .......... .......... .......... .......... 31% 224M 0s 2019-11-29T11:27:34.541 INFO:teuthology.orchestra.run.smithi069.stderr: 23300K .......... .......... .......... .......... .......... 31% 194M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23350K .......... .......... .......... .......... .......... 31% 177M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23400K .......... .......... .......... .......... .......... 31% 210M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23450K .......... .......... .......... .......... .......... 31% 202M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23500K .......... .......... .......... .......... .......... 32% 213M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23550K .......... .......... .......... .......... .......... 32% 185M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23600K .......... .......... .......... .......... .......... 32% 224M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23650K .......... .......... .......... .......... .......... 32% 231M 0s 2019-11-29T11:27:34.542 INFO:teuthology.orchestra.run.smithi069.stderr: 23700K .......... .......... .......... .......... .......... 32% 217M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 23750K .......... .......... .......... .......... .......... 32% 201M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 23800K .......... .......... .......... .......... .......... 32% 228M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 23850K .......... .......... .......... .......... .......... 32% 208M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 23900K .......... .......... .......... .......... .......... 32% 219M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 23950K .......... .......... .......... .......... .......... 32% 194M 0s 2019-11-29T11:27:34.543 INFO:teuthology.orchestra.run.smithi069.stderr: 24000K .......... .......... .......... .......... .......... 32% 229M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24050K .......... .......... .......... .......... .......... 32% 222M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24100K .......... .......... .......... .......... .......... 32% 221M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24150K .......... .......... .......... .......... .......... 32% 200M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24200K .......... .......... .......... .......... .......... 32% 226M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24250K .......... .......... .......... .......... .......... 33% 214M 0s 2019-11-29T11:27:34.544 INFO:teuthology.orchestra.run.smithi069.stderr: 24300K .......... .......... .......... .......... .......... 33% 222M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24350K .......... .......... .......... .......... .......... 33% 196M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24400K .......... .......... .......... .......... .......... 33% 224M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24450K .......... .......... .......... .......... .......... 33% 200M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24500K .......... .......... .......... .......... .......... 33% 203M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24550K .......... .......... .......... .......... .......... 33% 182M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24600K .......... .......... .......... .......... .......... 33% 214M 0s 2019-11-29T11:27:34.545 INFO:teuthology.orchestra.run.smithi069.stderr: 24650K .......... .......... .......... .......... .......... 33% 210M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24700K .......... .......... .......... .......... .......... 33% 207M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24750K .......... .......... .......... .......... .......... 33% 166M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24800K .......... .......... .......... .......... .......... 33% 210M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24850K .......... .......... .......... .......... .......... 33% 217M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24900K .......... .......... .......... .......... .......... 33% 219M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 24950K .......... .......... .......... .......... .......... 33% 196M 0s 2019-11-29T11:27:34.546 INFO:teuthology.orchestra.run.smithi069.stderr: 25000K .......... .......... .......... .......... .......... 34% 212M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25050K .......... .......... .......... .......... .......... 34% 208M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25100K .......... .......... .......... .......... .......... 34% 208M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25150K .......... .......... .......... .......... .......... 34% 173M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25200K .......... .......... .......... .......... .......... 34% 201M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25250K .......... .......... .......... .......... .......... 34% 213M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25300K .......... .......... .......... .......... .......... 34% 212M 0s 2019-11-29T11:27:34.547 INFO:teuthology.orchestra.run.smithi069.stderr: 25350K .......... .......... .......... .......... .......... 34% 168M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25400K .......... .......... .......... .......... .......... 34% 218M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25450K .......... .......... .......... .......... .......... 34% 211M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25500K .......... .......... .......... .......... .......... 34% 211M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25550K .......... .......... .......... .......... .......... 34% 202M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25600K .......... .......... .......... .......... .......... 34% 220M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25650K .......... .......... .......... .......... .......... 34% 211M 0s 2019-11-29T11:27:34.548 INFO:teuthology.orchestra.run.smithi069.stderr: 25700K .......... .......... .......... .......... .......... 35% 213M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 25750K .......... .......... .......... .......... .......... 35% 176M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 25800K .......... .......... .......... .......... .......... 35% 212M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 25850K .......... .......... .......... .......... .......... 35% 226M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 25900K .......... .......... .......... .......... .......... 35% 204M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 25950K .......... .......... .......... .......... .......... 35% 193M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 26000K .......... .......... .......... .......... .......... 35% 231M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 26050K .......... .......... .......... .......... .......... 35% 213M 0s 2019-11-29T11:27:34.549 INFO:teuthology.orchestra.run.smithi069.stderr: 26100K .......... .......... .......... .......... .......... 35% 224M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26150K .......... .......... .......... .......... .......... 35% 187M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26200K .......... .......... .......... .......... .......... 35% 228M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26250K .......... .......... .......... .......... .......... 35% 219M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26300K .......... .......... .......... .......... .......... 35% 228M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26350K .......... .......... .......... .......... .......... 35% 200M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26400K .......... .......... .......... .......... .......... 35% 233M 0s 2019-11-29T11:27:34.550 INFO:teuthology.orchestra.run.smithi069.stderr: 26450K .......... .......... .......... .......... .......... 36% 217M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26500K .......... .......... .......... .......... .......... 36% 222M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26550K .......... .......... .......... .......... .......... 36% 184M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26600K .......... .......... .......... .......... .......... 36% 222M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26650K .......... .......... .......... .......... .......... 36% 218M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26700K .......... .......... .......... .......... .......... 36% 196M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26750K .......... .......... .......... .......... .......... 36% 191M 0s 2019-11-29T11:27:34.551 INFO:teuthology.orchestra.run.smithi069.stderr: 26800K .......... .......... .......... .......... .......... 36% 214M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 26850K .......... .......... .......... .......... .......... 36% 225M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 26900K .......... .......... .......... .......... .......... 36% 205M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 26950K .......... .......... .......... .......... .......... 36% 184M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 27000K .......... .......... .......... .......... .......... 36% 228M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 27050K .......... .......... .......... .......... .......... 36% 226M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 27100K .......... .......... .......... .......... .......... 36% 219M 0s 2019-11-29T11:27:34.552 INFO:teuthology.orchestra.run.smithi069.stderr: 27150K .......... .......... .......... .......... .......... 36% 205M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27200K .......... .......... .......... .......... .......... 37% 247M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27250K .......... .......... .......... .......... .......... 37% 243M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27300K .......... .......... .......... .......... .......... 37% 232M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27350K .......... .......... .......... .......... .......... 37% 192M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27400K .......... .......... .......... .......... .......... 37% 231M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27450K .......... .......... .......... .......... .......... 37% 234M 0s 2019-11-29T11:27:34.553 INFO:teuthology.orchestra.run.smithi069.stderr: 27500K .......... .......... .......... .......... .......... 37% 226M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27550K .......... .......... .......... .......... .......... 37% 185M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27600K .......... .......... .......... .......... .......... 37% 233M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27650K .......... .......... .......... .......... .......... 37% 216M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27700K .......... .......... .......... .......... .......... 37% 218M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27750K .......... .......... .......... .......... .......... 37% 182M 0s 2019-11-29T11:27:34.554 INFO:teuthology.orchestra.run.smithi069.stderr: 27800K .......... .......... .......... .......... .......... 37% 221M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 27850K .......... .......... .......... .......... .......... 37% 237M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 27900K .......... .......... .......... .......... .......... 37% 227M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 27950K .......... .......... .......... .......... .......... 38% 215M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 28000K .......... .......... .......... .......... .......... 38% 233M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 28050K .......... .......... .......... .......... .......... 38% 240M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 28100K .......... .......... .......... .......... .......... 38% 237M 0s 2019-11-29T11:27:34.555 INFO:teuthology.orchestra.run.smithi069.stderr: 28150K .......... .......... .......... .......... .......... 38% 196M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28200K .......... .......... .......... .......... .......... 38% 242M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28250K .......... .......... .......... .......... .......... 38% 233M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28300K .......... .......... .......... .......... .......... 38% 243M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28350K .......... .......... .......... .......... .......... 38% 212M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28400K .......... .......... .......... .......... .......... 38% 229M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28450K .......... .......... .......... .......... .......... 38% 223M 0s 2019-11-29T11:27:34.556 INFO:teuthology.orchestra.run.smithi069.stderr: 28500K .......... .......... .......... .......... .......... 38% 241M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28550K .......... .......... .......... .......... .......... 38% 205M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28600K .......... .......... .......... .......... .......... 38% 282M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28650K .......... .......... .......... .......... .......... 39% 306M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28700K .......... .......... .......... .......... .......... 39% 327M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28750K .......... .......... .......... .......... .......... 39% 253M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28800K .......... .......... .......... .......... .......... 39% 338M 0s 2019-11-29T11:27:34.557 INFO:teuthology.orchestra.run.smithi069.stderr: 28850K .......... .......... .......... .......... .......... 39% 327M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 28900K .......... .......... .......... .......... .......... 39% 334M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 28950K .......... .......... .......... .......... .......... 39% 293M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29000K .......... .......... .......... .......... .......... 39% 348M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29050K .......... .......... .......... .......... .......... 39% 347M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29100K .......... .......... .......... .......... .......... 39% 333M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29150K .......... .......... .......... .......... .......... 39% 281M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29200K .......... .......... .......... .......... .......... 39% 333M 0s 2019-11-29T11:27:34.558 INFO:teuthology.orchestra.run.smithi069.stderr: 29250K .......... .......... .......... .......... .......... 39% 337M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29300K .......... .......... .......... .......... .......... 39% 306M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29350K .......... .......... .......... .......... .......... 39% 280M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29400K .......... .......... .......... .......... .......... 40% 322M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29450K .......... .......... .......... .......... .......... 40% 347M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29500K .......... .......... .......... .......... .......... 40% 331M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29550K .......... .......... .......... .......... .......... 40% 295M 0s 2019-11-29T11:27:34.559 INFO:teuthology.orchestra.run.smithi069.stderr: 29600K .......... .......... .......... .......... .......... 40% 321M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29650K .......... .......... .......... .......... .......... 40% 320M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29700K .......... .......... .......... .......... .......... 40% 311M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29750K .......... .......... .......... .......... .......... 40% 256M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29800K .......... .......... .......... .......... .......... 40% 320M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29850K .......... .......... .......... .......... .......... 40% 311M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29900K .......... .......... .......... .......... .......... 40% 331M 0s 2019-11-29T11:27:34.560 INFO:teuthology.orchestra.run.smithi069.stderr: 29950K .......... .......... .......... .......... .......... 40% 288M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30000K .......... .......... .......... .......... .......... 40% 335M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30050K .......... .......... .......... .......... .......... 40% 282M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30100K .......... .......... .......... .......... .......... 40% 350M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30150K .......... .......... .......... .......... .......... 41% 313M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30200K .......... .......... .......... .......... .......... 41% 345M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30250K .......... .......... .......... .......... .......... 41% 343M 0s 2019-11-29T11:27:34.561 INFO:teuthology.orchestra.run.smithi069.stderr: 30300K .......... .......... .......... .......... .......... 41% 337M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30350K .......... .......... .......... .......... .......... 41% 304M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30400K .......... .......... .......... .......... .......... 41% 341M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30450K .......... .......... .......... .......... .......... 41% 331M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30500K .......... .......... .......... .......... .......... 41% 344M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30550K .......... .......... .......... .......... .......... 41% 289M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30600K .......... .......... .......... .......... .......... 41% 330M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30650K .......... .......... .......... .......... .......... 41% 351M 0s 2019-11-29T11:27:34.562 INFO:teuthology.orchestra.run.smithi069.stderr: 30700K .......... .......... .......... .......... .......... 41% 356M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 30750K .......... .......... .......... .......... .......... 41% 286M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 30800K .......... .......... .......... .......... .......... 41% 358M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 30850K .......... .......... .......... .......... .......... 42% 358M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 30900K .......... .......... .......... .......... .......... 42% 360M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 30950K .......... .......... .......... .......... .......... 42% 307M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 31000K .......... .......... .......... .......... .......... 42% 362M 0s 2019-11-29T11:27:34.563 INFO:teuthology.orchestra.run.smithi069.stderr: 31050K .......... .......... .......... .......... .......... 42% 346M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31100K .......... .......... .......... .......... .......... 42% 346M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31150K .......... .......... .......... .......... .......... 42% 294M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31200K .......... .......... .......... .......... .......... 42% 359M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31250K .......... .......... .......... .......... .......... 42% 341M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31300K .......... .......... .......... .......... .......... 42% 356M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31350K .......... .......... .......... .......... .......... 42% 303M 0s 2019-11-29T11:27:34.564 INFO:teuthology.orchestra.run.smithi069.stderr: 31400K .......... .......... .......... .......... .......... 42% 294M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31450K .......... .......... .......... .......... .......... 42% 301M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31500K .......... .......... .......... .......... .......... 42% 340M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31550K .......... .......... .......... .......... .......... 42% 294M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31600K .......... .......... .......... .......... .......... 43% 340M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31650K .......... .......... .......... .......... .......... 43% 285M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31700K .......... .......... .......... .......... .......... 43% 301M 0s 2019-11-29T11:27:34.565 INFO:teuthology.orchestra.run.smithi069.stderr: 31750K .......... .......... .......... .......... .......... 43% 237M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 31800K .......... .......... .......... .......... .......... 43% 312M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 31850K .......... .......... .......... .......... .......... 43% 249M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 31900K .......... .......... .......... .......... .......... 43% 279M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 31950K .......... .......... .......... .......... .......... 43% 276M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 32000K .......... .......... .......... .......... .......... 43% 293M 0s 2019-11-29T11:27:34.566 INFO:teuthology.orchestra.run.smithi069.stderr: 32050K .......... .......... .......... .......... .......... 43% 313M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32100K .......... .......... .......... .......... .......... 43% 358M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32150K .......... .......... .......... .......... .......... 43% 305M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32200K .......... .......... .......... .......... .......... 43% 316M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32250K .......... .......... .......... .......... .......... 43% 343M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32300K .......... .......... .......... .......... .......... 43% 286M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32350K .......... .......... .......... .......... .......... 44% 242M 0s 2019-11-29T11:27:34.567 INFO:teuthology.orchestra.run.smithi069.stderr: 32400K .......... .......... .......... .......... .......... 44% 297M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32450K .......... .......... .......... .......... .......... 44% 323M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32500K .......... .......... .......... .......... .......... 44% 335M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32550K .......... .......... .......... .......... .......... 44% 287M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32600K .......... .......... .......... .......... .......... 44% 249M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32650K .......... .......... .......... .......... .......... 44% 309M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32700K .......... .......... .......... .......... .......... 44% 295M 0s 2019-11-29T11:27:34.568 INFO:teuthology.orchestra.run.smithi069.stderr: 32750K .......... .......... .......... .......... .......... 44% 294M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 32800K .......... .......... .......... .......... .......... 44% 364M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 32850K .......... .......... .......... .......... .......... 44% 322M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 32900K .......... .......... .......... .......... .......... 44% 342M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 32950K .......... .......... .......... .......... .......... 44% 248M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 33000K .......... .......... .......... .......... .......... 44% 272M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 33050K .......... .......... .......... .......... .......... 44% 299M 0s 2019-11-29T11:27:34.569 INFO:teuthology.orchestra.run.smithi069.stderr: 33100K .......... .......... .......... .......... .......... 45% 355M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33150K .......... .......... .......... .......... .......... 45% 246M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33200K .......... .......... .......... .......... .......... 45% 283M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33250K .......... .......... .......... .......... .......... 45% 295M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33300K .......... .......... .......... .......... .......... 45% 306M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33350K .......... .......... .......... .......... .......... 45% 261M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33400K .......... .......... .......... .......... .......... 45% 284M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33450K .......... .......... .......... .......... .......... 45% 329M 0s 2019-11-29T11:27:34.570 INFO:teuthology.orchestra.run.smithi069.stderr: 33500K .......... .......... .......... .......... .......... 45% 293M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33550K .......... .......... .......... .......... .......... 45% 234M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33600K .......... .......... .......... .......... .......... 45% 260M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33650K .......... .......... .......... .......... .......... 45% 328M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33700K .......... .......... .......... .......... .......... 45% 332M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33750K .......... .......... .......... .......... .......... 45% 262M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33800K .......... .......... .......... .......... .......... 46% 268M 0s 2019-11-29T11:27:34.571 INFO:teuthology.orchestra.run.smithi069.stderr: 33850K .......... .......... .......... .......... .......... 46% 323M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 33900K .......... .......... .......... .......... .......... 46% 340M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 33950K .......... .......... .......... .......... .......... 46% 267M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 34000K .......... .......... .......... .......... .......... 46% 284M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 34050K .......... .......... .......... .......... .......... 46% 323M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 34100K .......... .......... .......... .......... .......... 46% 296M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 34150K .......... .......... .......... .......... .......... 46% 274M 0s 2019-11-29T11:27:34.572 INFO:teuthology.orchestra.run.smithi069.stderr: 34200K .......... .......... .......... .......... .......... 46% 354M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34250K .......... .......... .......... .......... .......... 46% 228M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34300K .......... .......... .......... .......... .......... 46% 262M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34350K .......... .......... .......... .......... .......... 46% 244M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34400K .......... .......... .......... .......... .......... 46% 279M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34450K .......... .......... .......... .......... .......... 46% 253M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34500K .......... .......... .......... .......... .......... 46% 253M 0s 2019-11-29T11:27:34.573 INFO:teuthology.orchestra.run.smithi069.stderr: 34550K .......... .......... .......... .......... .......... 47% 212M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34600K .......... .......... .......... .......... .......... 47% 254M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34650K .......... .......... .......... .......... .......... 47% 270M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34700K .......... .......... .......... .......... .......... 47% 303M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34750K .......... .......... .......... .......... .......... 47% 263M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34800K .......... .......... .......... .......... .......... 47% 300M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34850K .......... .......... .......... .......... .......... 47% 299M 0s 2019-11-29T11:27:34.574 INFO:teuthology.orchestra.run.smithi069.stderr: 34900K .......... .......... .......... .......... .......... 47% 259M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 34950K .......... .......... .......... .......... .......... 47% 254M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35000K .......... .......... .......... .......... .......... 47% 278M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35050K .......... .......... .......... .......... .......... 47% 276M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35100K .......... .......... .......... .......... .......... 47% 274M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35150K .......... .......... .......... .......... .......... 47% 250M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35200K .......... .......... .......... .......... .......... 47% 296M 0s 2019-11-29T11:27:34.575 INFO:teuthology.orchestra.run.smithi069.stderr: 35250K .......... .......... .......... .......... .......... 47% 306M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35300K .......... .......... .......... .......... .......... 48% 267M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35350K .......... .......... .......... .......... .......... 48% 253M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35400K .......... .......... .......... .......... .......... 48% 291M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35450K .......... .......... .......... .......... .......... 48% 276M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35500K .......... .......... .......... .......... .......... 48% 288M 0s 2019-11-29T11:27:34.576 INFO:teuthology.orchestra.run.smithi069.stderr: 35550K .......... .......... .......... .......... .......... 48% 241M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35600K .......... .......... .......... .......... .......... 48% 291M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35650K .......... .......... .......... .......... .......... 48% 291M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35700K .......... .......... .......... .......... .......... 48% 270M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35750K .......... .......... .......... .......... .......... 48% 240M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35800K .......... .......... .......... .......... .......... 48% 250M 0s 2019-11-29T11:27:34.577 INFO:teuthology.orchestra.run.smithi069.stderr: 35850K .......... .......... .......... .......... .......... 48% 285M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 35900K .......... .......... .......... .......... .......... 48% 287M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 35950K .......... .......... .......... .......... .......... 48% 259M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 36000K .......... .......... .......... .......... .......... 49% 281M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 36050K .......... .......... .......... .......... .......... 49% 271M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 36100K .......... .......... .......... .......... .......... 49% 302M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 36150K .......... .......... .......... .......... .......... 49% 244M 0s 2019-11-29T11:27:34.578 INFO:teuthology.orchestra.run.smithi069.stderr: 36200K .......... .......... .......... .......... .......... 49% 313M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36250K .......... .......... .......... .......... .......... 49% 274M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36300K .......... .......... .......... .......... .......... 49% 303M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36350K .......... .......... .......... .......... .......... 49% 237M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36400K .......... .......... .......... .......... .......... 49% 291M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36450K .......... .......... .......... .......... .......... 49% 271M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36500K .......... .......... .......... .......... .......... 49% 295M 0s 2019-11-29T11:27:34.579 INFO:teuthology.orchestra.run.smithi069.stderr: 36550K .......... .......... .......... .......... .......... 49% 237M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36600K .......... .......... .......... .......... .......... 49% 266M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36650K .......... .......... .......... .......... .......... 49% 268M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36700K .......... .......... .......... .......... .......... 49% 310M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36750K .......... .......... .......... .......... .......... 50% 248M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36800K .......... .......... .......... .......... .......... 50% 283M 0s 2019-11-29T11:27:34.580 INFO:teuthology.orchestra.run.smithi069.stderr: 36850K .......... .......... .......... .......... .......... 50% 296M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 36900K .......... .......... .......... .......... .......... 50% 307M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 36950K .......... .......... .......... .......... .......... 50% 277M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 37000K .......... .......... .......... .......... .......... 50% 322M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 37050K .......... .......... .......... .......... .......... 50% 270M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 37100K .......... .......... .......... .......... .......... 50% 280M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 37150K .......... .......... .......... .......... .......... 50% 220M 0s 2019-11-29T11:27:34.581 INFO:teuthology.orchestra.run.smithi069.stderr: 37200K .......... .......... .......... .......... .......... 50% 290M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37250K .......... .......... .......... .......... .......... 50% 238M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37300K .......... .......... .......... .......... .......... 50% 271M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37350K .......... .......... .......... .......... .......... 50% 231M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37400K .......... .......... .......... .......... .......... 50% 262M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37450K .......... .......... .......... .......... .......... 50% 273M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37500K .......... .......... .......... .......... .......... 51% 255M 0s 2019-11-29T11:27:34.582 INFO:teuthology.orchestra.run.smithi069.stderr: 37550K .......... .......... .......... .......... .......... 51% 255M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37600K .......... .......... .......... .......... .......... 51% 302M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37650K .......... .......... .......... .......... .......... 51% 285M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37700K .......... .......... .......... .......... .......... 51% 266M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37750K .......... .......... .......... .......... .......... 51% 251M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37800K .......... .......... .......... .......... .......... 51% 300M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37850K .......... .......... .......... .......... .......... 51% 304M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37900K .......... .......... .......... .......... .......... 51% 328M 0s 2019-11-29T11:27:34.583 INFO:teuthology.orchestra.run.smithi069.stderr: 37950K .......... .......... .......... .......... .......... 51% 325M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38000K .......... .......... .......... .......... .......... 51% 371M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38050K .......... .......... .......... .......... .......... 51% 377M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38100K .......... .......... .......... .......... .......... 51% 369M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38150K .......... .......... .......... .......... .......... 51% 307M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38200K .......... .......... .......... .......... .......... 51% 385M 0s 2019-11-29T11:27:34.584 INFO:teuthology.orchestra.run.smithi069.stderr: 38250K .......... .......... .......... .......... .......... 52% 356M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38300K .......... .......... .......... .......... .......... 52% 382M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38350K .......... .......... .......... .......... .......... 52% 295M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38400K .......... .......... .......... .......... .......... 52% 308M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38450K .......... .......... .......... .......... .......... 52% 305M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38500K .......... .......... .......... .......... .......... 52% 345M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38550K .......... .......... .......... .......... .......... 52% 303M 0s 2019-11-29T11:27:34.585 INFO:teuthology.orchestra.run.smithi069.stderr: 38600K .......... .......... .......... .......... .......... 52% 359M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38650K .......... .......... .......... .......... .......... 52% 348M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38700K .......... .......... .......... .......... .......... 52% 339M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38750K .......... .......... .......... .......... .......... 52% 315M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38800K .......... .......... .......... .......... .......... 52% 377M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38850K .......... .......... .......... .......... .......... 52% 359M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38900K .......... .......... .......... .......... .......... 52% 385M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 38950K .......... .......... .......... .......... .......... 53% 268M 0s 2019-11-29T11:27:34.586 INFO:teuthology.orchestra.run.smithi069.stderr: 39000K .......... .......... .......... .......... .......... 53% 313M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39050K .......... .......... .......... .......... .......... 53% 347M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39100K .......... .......... .......... .......... .......... 53% 277M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39150K .......... .......... .......... .......... .......... 53% 267M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39200K .......... .......... .......... .......... .......... 53% 360M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39250K .......... .......... .......... .......... .......... 53% 342M 0s 2019-11-29T11:27:34.587 INFO:teuthology.orchestra.run.smithi069.stderr: 39300K .......... .......... .......... .......... .......... 53% 333M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39350K .......... .......... .......... .......... .......... 53% 325M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39400K .......... .......... .......... .......... .......... 53% 368M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39450K .......... .......... .......... .......... .......... 53% 377M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39500K .......... .......... .......... .......... .......... 53% 377M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39550K .......... .......... .......... .......... .......... 53% 312M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39600K .......... .......... .......... .......... .......... 53% 374M 0s 2019-11-29T11:27:34.588 INFO:teuthology.orchestra.run.smithi069.stderr: 39650K .......... .......... .......... .......... .......... 53% 370M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39700K .......... .......... .......... .......... .......... 54% 369M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39750K .......... .......... .......... .......... .......... 54% 266M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39800K .......... .......... .......... .......... .......... 54% 277M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39850K .......... .......... .......... .......... .......... 54% 326M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39900K .......... .......... .......... .......... .......... 54% 359M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 39950K .......... .......... .......... .......... .......... 54% 269M 0s 2019-11-29T11:27:34.589 INFO:teuthology.orchestra.run.smithi069.stderr: 40000K .......... .......... .......... .......... .......... 54% 301M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40050K .......... .......... .......... .......... .......... 54% 310M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40100K .......... .......... .......... .......... .......... 54% 359M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40150K .......... .......... .......... .......... .......... 54% 244M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40200K .......... .......... .......... .......... .......... 54% 326M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40250K .......... .......... .......... .......... .......... 54% 317M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40300K .......... .......... .......... .......... .......... 54% 339M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40350K .......... .......... .......... .......... .......... 54% 310M 0s 2019-11-29T11:27:34.590 INFO:teuthology.orchestra.run.smithi069.stderr: 40400K .......... .......... .......... .......... .......... 54% 371M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40450K .......... .......... .......... .......... .......... 55% 381M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40500K .......... .......... .......... .......... .......... 55% 336M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40550K .......... .......... .......... .......... .......... 55% 273M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40600K .......... .......... .......... .......... .......... 55% 318M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40650K .......... .......... .......... .......... .......... 55% 349M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40700K .......... .......... .......... .......... .......... 55% 340M 0s 2019-11-29T11:27:34.591 INFO:teuthology.orchestra.run.smithi069.stderr: 40750K .......... .......... .......... .......... .......... 55% 280M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 40800K .......... .......... .......... .......... .......... 55% 322M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 40850K .......... .......... .......... .......... .......... 55% 367M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 40900K .......... .......... .......... .......... .......... 55% 350M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 40950K .......... .......... .......... .......... .......... 55% 296M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 41000K .......... .......... .......... .......... .......... 55% 357M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 41050K .......... .......... .......... .......... .......... 55% 381M 0s 2019-11-29T11:27:34.592 INFO:teuthology.orchestra.run.smithi069.stderr: 41100K .......... .......... .......... .......... .......... 55% 386M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41150K .......... .......... .......... .......... .......... 56% 351M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41200K .......... .......... .......... .......... .......... 56% 390M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41250K .......... .......... .......... .......... .......... 56% 371M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41300K .......... .......... .......... .......... .......... 56% 373M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41350K .......... .......... .......... .......... .......... 56% 309M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41400K .......... .......... .......... .......... .......... 56% 316M 0s 2019-11-29T11:27:34.593 INFO:teuthology.orchestra.run.smithi069.stderr: 41450K .......... .......... .......... .......... .......... 56% 228M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41500K .......... .......... .......... .......... .......... 56% 215M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41550K .......... .......... .......... .......... .......... 56% 188M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41600K .......... .......... .......... .......... .......... 56% 211M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41650K .......... .......... .......... .......... .......... 56% 236M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41700K .......... .......... .......... .......... .......... 56% 273M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41750K .......... .......... .......... .......... .......... 56% 228M 0s 2019-11-29T11:27:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: 41800K .......... .......... .......... .......... .......... 56% 206M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 41850K .......... .......... .......... .......... .......... 56% 267M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 41900K .......... .......... .......... .......... .......... 57% 348M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 41950K .......... .......... .......... .......... .......... 57% 248M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 42000K .......... .......... .......... .......... .......... 57% 355M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 42050K .......... .......... .......... .......... .......... 57% 373M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 42100K .......... .......... .......... .......... .......... 57% 383M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 42150K .......... .......... .......... .......... .......... 57% 337M 0s 2019-11-29T11:27:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: 42200K .......... .......... .......... .......... .......... 57% 310M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42250K .......... .......... .......... .......... .......... 57% 201M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42300K .......... .......... .......... .......... .......... 57% 319M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42350K .......... .......... .......... .......... .......... 57% 353M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42400K .......... .......... .......... .......... .......... 57% 378M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42450K .......... .......... .......... .......... .......... 57% 282M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42500K .......... .......... .......... .......... .......... 57% 285M 0s 2019-11-29T11:27:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: 42550K .......... .......... .......... .......... .......... 57% 296M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42600K .......... .......... .......... .......... .......... 57% 309M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42650K .......... .......... .......... .......... .......... 58% 289M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42700K .......... .......... .......... .......... .......... 58% 298M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42750K .......... .......... .......... .......... .......... 58% 264M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42800K .......... .......... .......... .......... .......... 58% 281M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42850K .......... .......... .......... .......... .......... 58% 219M 0s 2019-11-29T11:27:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 42900K .......... .......... .......... .......... .......... 58% 253M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 42950K .......... .......... .......... .......... .......... 58% 274M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43000K .......... .......... .......... .......... .......... 58% 227M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43050K .......... .......... .......... .......... .......... 58% 188M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43100K .......... .......... .......... .......... .......... 58% 196M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43150K .......... .......... .......... .......... .......... 58% 262M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43200K .......... .......... .......... .......... .......... 58% 318M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43250K .......... .......... .......... .......... .......... 58% 370M 0s 2019-11-29T11:27:34.598 INFO:teuthology.orchestra.run.smithi069.stderr: 43300K .......... .......... .......... .......... .......... 58% 301M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43350K .......... .......... .......... .......... .......... 59% 318M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43400K .......... .......... .......... .......... .......... 59% 299M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43450K .......... .......... .......... .......... .......... 59% 184M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43500K .......... .......... .......... .......... .......... 59% 215M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43550K .......... .......... .......... .......... .......... 59% 230M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43600K .......... .......... .......... .......... .......... 59% 388M 0s 2019-11-29T11:27:34.599 INFO:teuthology.orchestra.run.smithi069.stderr: 43650K .......... .......... .......... .......... .......... 59% 356M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43700K .......... .......... .......... .......... .......... 59% 184M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43750K .......... .......... .......... .......... .......... 59% 216M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43800K .......... .......... .......... .......... .......... 59% 294M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43850K .......... .......... .......... .......... .......... 59% 285M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43900K .......... .......... .......... .......... .......... 59% 299M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 43950K .......... .......... .......... .......... .......... 59% 261M 0s 2019-11-29T11:27:34.600 INFO:teuthology.orchestra.run.smithi069.stderr: 44000K .......... .......... .......... .......... .......... 59% 289M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44050K .......... .......... .......... .......... .......... 59% 303M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44100K .......... .......... .......... .......... .......... 60% 292M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44150K .......... .......... .......... .......... .......... 60% 258M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44200K .......... .......... .......... .......... .......... 60% 336M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44250K .......... .......... .......... .......... .......... 60% 331M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44300K .......... .......... .......... .......... .......... 60% 302M 0s 2019-11-29T11:27:34.601 INFO:teuthology.orchestra.run.smithi069.stderr: 44350K .......... .......... .......... .......... .......... 60% 276M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44400K .......... .......... .......... .......... .......... 60% 256M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44450K .......... .......... .......... .......... .......... 60% 302M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44500K .......... .......... .......... .......... .......... 60% 336M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44550K .......... .......... .......... .......... .......... 60% 341M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44600K .......... .......... .......... .......... .......... 60% 366M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44650K .......... .......... .......... .......... .......... 60% 355M 0s 2019-11-29T11:27:34.602 INFO:teuthology.orchestra.run.smithi069.stderr: 44700K .......... .......... .......... .......... .......... 60% 330M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 44750K .......... .......... .......... .......... .......... 60% 250M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 44800K .......... .......... .......... .......... .......... 60% 332M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 44850K .......... .......... .......... .......... .......... 61% 360M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 44900K .......... .......... .......... .......... .......... 61% 323M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 44950K .......... .......... .......... .......... .......... 61% 309M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 45000K .......... .......... .......... .......... .......... 61% 360M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 45050K .......... .......... .......... .......... .......... 61% 387M 0s 2019-11-29T11:27:34.603 INFO:teuthology.orchestra.run.smithi069.stderr: 45100K .......... .......... .......... .......... .......... 61% 337M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45150K .......... .......... .......... .......... .......... 61% 297M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45200K .......... .......... .......... .......... .......... 61% 393M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45250K .......... .......... .......... .......... .......... 61% 384M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45300K .......... .......... .......... .......... .......... 61% 391M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45350K .......... .......... .......... .......... .......... 61% 346M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45400K .......... .......... .......... .......... .......... 61% 396M 0s 2019-11-29T11:27:34.604 INFO:teuthology.orchestra.run.smithi069.stderr: 45450K .......... .......... .......... .......... .......... 61% 343M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45500K .......... .......... .......... .......... .......... 61% 362M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45550K .......... .......... .......... .......... .......... 61% 281M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45600K .......... .......... .......... .......... .......... 62% 367M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45650K .......... .......... .......... .......... .......... 62% 370M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45700K .......... .......... .......... .......... .......... 62% 326M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45750K .......... .......... .......... .......... .......... 62% 286M 0s 2019-11-29T11:27:34.605 INFO:teuthology.orchestra.run.smithi069.stderr: 45800K .......... .......... .......... .......... .......... 62% 302M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 45850K .......... .......... .......... .......... .......... 62% 320M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 45900K .......... .......... .......... .......... .......... 62% 308M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 45950K .......... .......... .......... .......... .......... 62% 256M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 46000K .......... .......... .......... .......... .......... 62% 297M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 46050K .......... .......... .......... .......... .......... 62% 320M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 46100K .......... .......... .......... .......... .......... 62% 294M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 46150K .......... .......... .......... .......... .......... 62% 278M 0s 2019-11-29T11:27:34.606 INFO:teuthology.orchestra.run.smithi069.stderr: 46200K .......... .......... .......... .......... .......... 62% 315M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46250K .......... .......... .......... .......... .......... 62% 329M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46300K .......... .......... .......... .......... .......... 63% 319M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46350K .......... .......... .......... .......... .......... 63% 241M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46400K .......... .......... .......... .......... .......... 63% 307M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46450K .......... .......... .......... .......... .......... 63% 339M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46500K .......... .......... .......... .......... .......... 63% 371M 0s 2019-11-29T11:27:34.607 INFO:teuthology.orchestra.run.smithi069.stderr: 46550K .......... .......... .......... .......... .......... 63% 286M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46600K .......... .......... .......... .......... .......... 63% 346M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46650K .......... .......... .......... .......... .......... 63% 408M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46700K .......... .......... .......... .......... .......... 63% 357M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46750K .......... .......... .......... .......... .......... 63% 312M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46800K .......... .......... .......... .......... .......... 63% 398M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46850K .......... .......... .......... .......... .......... 63% 380M 0s 2019-11-29T11:27:34.608 INFO:teuthology.orchestra.run.smithi069.stderr: 46900K .......... .......... .......... .......... .......... 63% 378M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 46950K .......... .......... .......... .......... .......... 63% 375M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 47000K .......... .......... .......... .......... .......... 63% 375M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 47050K .......... .......... .......... .......... .......... 64% 399M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 47100K .......... .......... .......... .......... .......... 64% 390M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 47150K .......... .......... .......... .......... .......... 64% 335M 0s 2019-11-29T11:27:34.609 INFO:teuthology.orchestra.run.smithi069.stderr: 47200K .......... .......... .......... .......... .......... 64% 399M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47250K .......... .......... .......... .......... .......... 64% 393M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47300K .......... .......... .......... .......... .......... 64% 392M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47350K .......... .......... .......... .......... .......... 64% 354M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47400K .......... .......... .......... .......... .......... 64% 380M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47450K .......... .......... .......... .......... .......... 64% 366M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47500K .......... .......... .......... .......... .......... 64% 408M 0s 2019-11-29T11:27:34.610 INFO:teuthology.orchestra.run.smithi069.stderr: 47550K .......... .......... .......... .......... .......... 64% 340M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47600K .......... .......... .......... .......... .......... 64% 400M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47650K .......... .......... .......... .......... .......... 64% 344M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47700K .......... .......... .......... .......... .......... 64% 387M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47750K .......... .......... .......... .......... .......... 64% 328M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47800K .......... .......... .......... .......... .......... 65% 395M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47850K .......... .......... .......... .......... .......... 65% 352M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47900K .......... .......... .......... .......... .......... 65% 326M 0s 2019-11-29T11:27:34.611 INFO:teuthology.orchestra.run.smithi069.stderr: 47950K .......... .......... .......... .......... .......... 65% 284M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48000K .......... .......... .......... .......... .......... 65% 369M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48050K .......... .......... .......... .......... .......... 65% 373M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48100K .......... .......... .......... .......... .......... 65% 410M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48150K .......... .......... .......... .......... .......... 65% 352M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48200K .......... .......... .......... .......... .......... 65% 421M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48250K .......... .......... .......... .......... .......... 65% 391M 0s 2019-11-29T11:27:34.612 INFO:teuthology.orchestra.run.smithi069.stderr: 48300K .......... .......... .......... .......... .......... 65% 372M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48350K .......... .......... .......... .......... .......... 65% 334M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48400K .......... .......... .......... .......... .......... 65% 400M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48450K .......... .......... .......... .......... .......... 65% 412M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48500K .......... .......... .......... .......... .......... 66% 413M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48550K .......... .......... .......... .......... .......... 66% 358M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48600K .......... .......... .......... .......... .......... 66% 372M 0s 2019-11-29T11:27:34.613 INFO:teuthology.orchestra.run.smithi069.stderr: 48650K .......... .......... .......... .......... .......... 66% 399M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48700K .......... .......... .......... .......... .......... 66% 415M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48750K .......... .......... .......... .......... .......... 66% 327M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48800K .......... .......... .......... .......... .......... 66% 400M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48850K .......... .......... .......... .......... .......... 66% 412M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48900K .......... .......... .......... .......... .......... 66% 422M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 48950K .......... .......... .......... .......... .......... 66% 379M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 49000K .......... .......... .......... .......... .......... 66% 400M 0s 2019-11-29T11:27:34.614 INFO:teuthology.orchestra.run.smithi069.stderr: 49050K .......... .......... .......... .......... .......... 66% 327M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49100K .......... .......... .......... .......... .......... 66% 309M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49150K .......... .......... .......... .......... .......... 66% 271M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49200K .......... .......... .......... .......... .......... 66% 327M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49250K .......... .......... .......... .......... .......... 67% 333M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49300K .......... .......... .......... .......... .......... 67% 322M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49350K .......... .......... .......... .......... .......... 67% 276M 0s 2019-11-29T11:27:34.615 INFO:teuthology.orchestra.run.smithi069.stderr: 49400K .......... .......... .......... .......... .......... 67% 325M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49450K .......... .......... .......... .......... .......... 67% 338M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49500K .......... .......... .......... .......... .......... 67% 352M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49550K .......... .......... .......... .......... .......... 67% 281M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49600K .......... .......... .......... .......... .......... 67% 298M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49650K .......... .......... .......... .......... .......... 67% 331M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49700K .......... .......... .......... .......... .......... 67% 299M 0s 2019-11-29T11:27:34.616 INFO:teuthology.orchestra.run.smithi069.stderr: 49750K .......... .......... .......... .......... .......... 67% 294M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 49800K .......... .......... .......... .......... .......... 67% 340M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 49850K .......... .......... .......... .......... .......... 67% 323M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 49900K .......... .......... .......... .......... .......... 67% 329M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 49950K .......... .......... .......... .......... .......... 67% 276M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 50000K .......... .......... .......... .......... .......... 68% 304M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 50050K .......... .......... .......... .......... .......... 68% 340M 0s 2019-11-29T11:27:34.617 INFO:teuthology.orchestra.run.smithi069.stderr: 50100K .......... .......... .......... .......... .......... 68% 318M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50150K .......... .......... .......... .......... .......... 68% 307M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50200K .......... .......... .......... .......... .......... 68% 337M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50250K .......... .......... .......... .......... .......... 68% 330M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50300K .......... .......... .......... .......... .......... 68% 340M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50350K .......... .......... .......... .......... .......... 68% 274M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50400K .......... .......... .......... .......... .......... 68% 311M 0s 2019-11-29T11:27:34.618 INFO:teuthology.orchestra.run.smithi069.stderr: 50450K .......... .......... .......... .......... .......... 68% 328M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50500K .......... .......... .......... .......... .......... 68% 324M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50550K .......... .......... .......... .......... .......... 68% 299M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50600K .......... .......... .......... .......... .......... 68% 331M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50650K .......... .......... .......... .......... .......... 68% 321M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50700K .......... .......... .......... .......... .......... 68% 333M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50750K .......... .......... .......... .......... .......... 69% 282M 0s 2019-11-29T11:27:34.619 INFO:teuthology.orchestra.run.smithi069.stderr: 50800K .......... .......... .......... .......... .......... 69% 338M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 50850K .......... .......... .......... .......... .......... 69% 334M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 50900K .......... .......... .......... .......... .......... 69% 349M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 50950K .......... .......... .......... .......... .......... 69% 295M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 51000K .......... .......... .......... .......... .......... 69% 330M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 51050K .......... .......... .......... .......... .......... 69% 323M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 51100K .......... .......... .......... .......... .......... 69% 320M 0s 2019-11-29T11:27:34.620 INFO:teuthology.orchestra.run.smithi069.stderr: 51150K .......... .......... .......... .......... .......... 69% 260M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51200K .......... .......... .......... .......... .......... 69% 327M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51250K .......... .......... .......... .......... .......... 69% 300M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51300K .......... .......... .......... .......... .......... 69% 297M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51350K .......... .......... .......... .......... .......... 69% 305M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51400K .......... .......... .......... .......... .......... 69% 339M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51450K .......... .......... .......... .......... .......... 70% 342M 0s 2019-11-29T11:27:34.621 INFO:teuthology.orchestra.run.smithi069.stderr: 51500K .......... .......... .......... .......... .......... 70% 314M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51550K .......... .......... .......... .......... .......... 70% 274M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51600K .......... .......... .......... .......... .......... 70% 336M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51650K .......... .......... .......... .......... .......... 70% 332M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51700K .......... .......... .......... .......... .......... 70% 338M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51750K .......... .......... .......... .......... .......... 70% 297M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51800K .......... .......... .......... .......... .......... 70% 332M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51850K .......... .......... .......... .......... .......... 70% 347M 0s 2019-11-29T11:27:34.622 INFO:teuthology.orchestra.run.smithi069.stderr: 51900K .......... .......... .......... .......... .......... 70% 412M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 51950K .......... .......... .......... .......... .......... 70% 214M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52000K .......... .......... .......... .......... .......... 70% 269M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52050K .......... .......... .......... .......... .......... 70% 380M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52100K .......... .......... .......... .......... .......... 70% 366M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52150K .......... .......... .......... .......... .......... 70% 381M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52200K .......... .......... .......... .......... .......... 71% 385M 0s 2019-11-29T11:27:34.623 INFO:teuthology.orchestra.run.smithi069.stderr: 52250K .......... .......... .......... .......... .......... 71% 398M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52300K .......... .......... .......... .......... .......... 71% 410M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52350K .......... .......... .......... .......... .......... 71% 350M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52400K .......... .......... .......... .......... .......... 71% 434M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52450K .......... .......... .......... .......... .......... 71% 414M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52500K .......... .......... .......... .......... .......... 71% 419M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52550K .......... .......... .......... .......... .......... 71% 375M 0s 2019-11-29T11:27:34.624 INFO:teuthology.orchestra.run.smithi069.stderr: 52600K .......... .......... .......... .......... .......... 71% 412M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52650K .......... .......... .......... .......... .......... 71% 429M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52700K .......... .......... .......... .......... .......... 71% 413M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52750K .......... .......... .......... .......... .......... 71% 355M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52800K .......... .......... .......... .......... .......... 71% 422M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52850K .......... .......... .......... .......... .......... 71% 423M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52900K .......... .......... .......... .......... .......... 71% 415M 0s 2019-11-29T11:27:34.625 INFO:teuthology.orchestra.run.smithi069.stderr: 52950K .......... .......... .......... .......... .......... 72% 346M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53000K .......... .......... .......... .......... .......... 72% 388M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53050K .......... .......... .......... .......... .......... 72% 397M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53100K .......... .......... .......... .......... .......... 72% 406M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53150K .......... .......... .......... .......... .......... 72% 311M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53200K .......... .......... .......... .......... .......... 72% 366M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53250K .......... .......... .......... .......... .......... 72% 346M 0s 2019-11-29T11:27:34.626 INFO:teuthology.orchestra.run.smithi069.stderr: 53300K .......... .......... .......... .......... .......... 72% 302M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53350K .......... .......... .......... .......... .......... 72% 257M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53400K .......... .......... .......... .......... .......... 72% 328M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53450K .......... .......... .......... .......... .......... 72% 318M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53500K .......... .......... .......... .......... .......... 72% 321M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53550K .......... .......... .......... .......... .......... 72% 289M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53600K .......... .......... .......... .......... .......... 72% 327M 0s 2019-11-29T11:27:34.627 INFO:teuthology.orchestra.run.smithi069.stderr: 53650K .......... .......... .......... .......... .......... 73% 377M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53700K .......... .......... .......... .......... .......... 73% 312M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53750K .......... .......... .......... .......... .......... 73% 241M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53800K .......... .......... .......... .......... .......... 73% 318M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53850K .......... .......... .......... .......... .......... 73% 288M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53900K .......... .......... .......... .......... .......... 73% 307M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 53950K .......... .......... .......... .......... .......... 73% 300M 0s 2019-11-29T11:27:34.628 INFO:teuthology.orchestra.run.smithi069.stderr: 54000K .......... .......... .......... .......... .......... 73% 290M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54050K .......... .......... .......... .......... .......... 73% 336M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54100K .......... .......... .......... .......... .......... 73% 326M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54150K .......... .......... .......... .......... .......... 73% 306M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54200K .......... .......... .......... .......... .......... 73% 347M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54250K .......... .......... .......... .......... .......... 73% 330M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54300K .......... .......... .......... .......... .......... 73% 351M 0s 2019-11-29T11:27:34.629 INFO:teuthology.orchestra.run.smithi069.stderr: 54350K .......... .......... .......... .......... .......... 73% 277M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54400K .......... .......... .......... .......... .......... 74% 319M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54450K .......... .......... .......... .......... .......... 74% 331M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54500K .......... .......... .......... .......... .......... 74% 343M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54550K .......... .......... .......... .......... .......... 74% 278M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54600K .......... .......... .......... .......... .......... 74% 338M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54650K .......... .......... .......... .......... .......... 74% 344M 0s 2019-11-29T11:27:34.630 INFO:teuthology.orchestra.run.smithi069.stderr: 54700K .......... .......... .......... .......... .......... 74% 319M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 54750K .......... .......... .......... .......... .......... 74% 291M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 54800K .......... .......... .......... .......... .......... 74% 352M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 54850K .......... .......... .......... .......... .......... 74% 332M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 54900K .......... .......... .......... .......... .......... 74% 315M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 54950K .......... .......... .......... .......... .......... 74% 290M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 55000K .......... .......... .......... .......... .......... 74% 334M 0s 2019-11-29T11:27:34.631 INFO:teuthology.orchestra.run.smithi069.stderr: 55050K .......... .......... .......... .......... .......... 74% 340M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55100K .......... .......... .......... .......... .......... 74% 323M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55150K .......... .......... .......... .......... .......... 75% 264M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55200K .......... .......... .......... .......... .......... 75% 290M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55250K .......... .......... .......... .......... .......... 75% 303M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55300K .......... .......... .......... .......... .......... 75% 360M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55350K .......... .......... .......... .......... .......... 75% 287M 0s 2019-11-29T11:27:34.632 INFO:teuthology.orchestra.run.smithi069.stderr: 55400K .......... .......... .......... .......... .......... 75% 317M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55450K .......... .......... .......... .......... .......... 75% 347M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55500K .......... .......... .......... .......... .......... 75% 332M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55550K .......... .......... .......... .......... .......... 75% 284M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55600K .......... .......... .......... .......... .......... 75% 329M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55650K .......... .......... .......... .......... .......... 75% 353M 0s 2019-11-29T11:27:34.633 INFO:teuthology.orchestra.run.smithi069.stderr: 55700K .......... .......... .......... .......... .......... 75% 336M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 55750K .......... .......... .......... .......... .......... 75% 267M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 55800K .......... .......... .......... .......... .......... 75% 329M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 55850K .......... .......... .......... .......... .......... 75% 333M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 55900K .......... .......... .......... .......... .......... 76% 337M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 55950K .......... .......... .......... .......... .......... 76% 286M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 56000K .......... .......... .......... .......... .......... 76% 322M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 56050K .......... .......... .......... .......... .......... 76% 325M 0s 2019-11-29T11:27:34.634 INFO:teuthology.orchestra.run.smithi069.stderr: 56100K .......... .......... .......... .......... .......... 76% 345M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56150K .......... .......... .......... .......... .......... 76% 296M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56200K .......... .......... .......... .......... .......... 76% 327M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56250K .......... .......... .......... .......... .......... 76% 323M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56300K .......... .......... .......... .......... .......... 76% 361M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56350K .......... .......... .......... .......... .......... 76% 282M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56400K .......... .......... .......... .......... .......... 76% 338M 0s 2019-11-29T11:27:34.635 INFO:teuthology.orchestra.run.smithi069.stderr: 56450K .......... .......... .......... .......... .......... 76% 330M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56500K .......... .......... .......... .......... .......... 76% 311M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56550K .......... .......... .......... .......... .......... 76% 276M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56600K .......... .......... .......... .......... .......... 77% 319M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56650K .......... .......... .......... .......... .......... 77% 341M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56700K .......... .......... .......... .......... .......... 77% 323M 0s 2019-11-29T11:27:34.636 INFO:teuthology.orchestra.run.smithi069.stderr: 56750K .......... .......... .......... .......... .......... 77% 293M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 56800K .......... .......... .......... .......... .......... 77% 263M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 56850K .......... .......... .......... .......... .......... 77% 281M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 56900K .......... .......... .......... .......... .......... 77% 287M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 56950K .......... .......... .......... .......... .......... 77% 245M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 57000K .......... .......... .......... .......... .......... 77% 279M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 57050K .......... .......... .......... .......... .......... 77% 274M 0s 2019-11-29T11:27:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: 57100K .......... .......... .......... .......... .......... 77% 286M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57150K .......... .......... .......... .......... .......... 77% 230M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57200K .......... .......... .......... .......... .......... 77% 270M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57250K .......... .......... .......... .......... .......... 77% 272M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57300K .......... .......... .......... .......... .......... 77% 279M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57350K .......... .......... .......... .......... .......... 78% 248M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57400K .......... .......... .......... .......... .......... 78% 276M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57450K .......... .......... .......... .......... .......... 78% 289M 0s 2019-11-29T11:27:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: 57500K .......... .......... .......... .......... .......... 78% 265M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57550K .......... .......... .......... .......... .......... 78% 231M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57600K .......... .......... .......... .......... .......... 78% 273M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57650K .......... .......... .......... .......... .......... 78% 275M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57700K .......... .......... .......... .......... .......... 78% 274M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57750K .......... .......... .......... .......... .......... 78% 246M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57800K .......... .......... .......... .......... .......... 78% 271M 0s 2019-11-29T11:27:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 57850K .......... .......... .......... .......... .......... 78% 274M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 57900K .......... .......... .......... .......... .......... 78% 281M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 57950K .......... .......... .......... .......... .......... 78% 241M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 58000K .......... .......... .......... .......... .......... 78% 290M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 58050K .......... .......... .......... .......... .......... 78% 283M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 58100K .......... .......... .......... .......... .......... 79% 273M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 58150K .......... .......... .......... .......... .......... 79% 257M 0s 2019-11-29T11:27:34.640 INFO:teuthology.orchestra.run.smithi069.stderr: 58200K .......... .......... .......... .......... .......... 79% 275M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58250K .......... .......... .......... .......... .......... 79% 295M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58300K .......... .......... .......... .......... .......... 79% 283M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58350K .......... .......... .......... .......... .......... 79% 246M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58400K .......... .......... .......... .......... .......... 79% 286M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58450K .......... .......... .......... .......... .......... 79% 281M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58500K .......... .......... .......... .......... .......... 79% 272M 0s 2019-11-29T11:27:34.641 INFO:teuthology.orchestra.run.smithi069.stderr: 58550K .......... .......... .......... .......... .......... 79% 232M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58600K .......... .......... .......... .......... .......... 79% 242M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58650K .......... .......... .......... .......... .......... 79% 275M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58700K .......... .......... .......... .......... .......... 79% 284M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58750K .......... .......... .......... .......... .......... 79% 228M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58800K .......... .......... .......... .......... .......... 80% 275M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58850K .......... .......... .......... .......... .......... 80% 290M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58900K .......... .......... .......... .......... .......... 80% 255M 0s 2019-11-29T11:27:34.642 INFO:teuthology.orchestra.run.smithi069.stderr: 58950K .......... .......... .......... .......... .......... 80% 234M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59000K .......... .......... .......... .......... .......... 80% 294M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59050K .......... .......... .......... .......... .......... 80% 269M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59100K .......... .......... .......... .......... .......... 80% 277M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59150K .......... .......... .......... .......... .......... 80% 224M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59200K .......... .......... .......... .......... .......... 80% 283M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59250K .......... .......... .......... .......... .......... 80% 280M 0s 2019-11-29T11:27:34.643 INFO:teuthology.orchestra.run.smithi069.stderr: 59300K .......... .......... .......... .......... .......... 80% 273M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59350K .......... .......... .......... .......... .......... 80% 249M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59400K .......... .......... .......... .......... .......... 80% 267M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59450K .......... .......... .......... .......... .......... 80% 262M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59500K .......... .......... .......... .......... .......... 80% 276M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59550K .......... .......... .......... .......... .......... 81% 248M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59600K .......... .......... .......... .......... .......... 81% 267M 0s 2019-11-29T11:27:34.644 INFO:teuthology.orchestra.run.smithi069.stderr: 59650K .......... .......... .......... .......... .......... 81% 288M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59700K .......... .......... .......... .......... .......... 81% 266M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59750K .......... .......... .......... .......... .......... 81% 239M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59800K .......... .......... .......... .......... .......... 81% 277M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59850K .......... .......... .......... .......... .......... 81% 285M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59900K .......... .......... .......... .......... .......... 81% 286M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 59950K .......... .......... .......... .......... .......... 81% 234M 0s 2019-11-29T11:27:34.645 INFO:teuthology.orchestra.run.smithi069.stderr: 60000K .......... .......... .......... .......... .......... 81% 297M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60050K .......... .......... .......... .......... .......... 81% 287M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60100K .......... .......... .......... .......... .......... 81% 277M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60150K .......... .......... .......... .......... .......... 81% 224M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60200K .......... .......... .......... .......... .......... 81% 291M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60250K .......... .......... .......... .......... .......... 81% 267M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60300K .......... .......... .......... .......... .......... 82% 252M 0s 2019-11-29T11:27:34.646 INFO:teuthology.orchestra.run.smithi069.stderr: 60350K .......... .......... .......... .......... .......... 82% 243M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60400K .......... .......... .......... .......... .......... 82% 260M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60450K .......... .......... .......... .......... .......... 82% 265M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60500K .......... .......... .......... .......... .......... 82% 268M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60550K .......... .......... .......... .......... .......... 82% 233M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60600K .......... .......... .......... .......... .......... 82% 260M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60650K .......... .......... .......... .......... .......... 82% 298M 0s 2019-11-29T11:27:34.647 INFO:teuthology.orchestra.run.smithi069.stderr: 60700K .......... .......... .......... .......... .......... 82% 289M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 60750K .......... .......... .......... .......... .......... 82% 236M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 60800K .......... .......... .......... .......... .......... 82% 283M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 60850K .......... .......... .......... .......... .......... 82% 284M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 60900K .......... .......... .......... .......... .......... 82% 258M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 60950K .......... .......... .......... .......... .......... 82% 260M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 61000K .......... .......... .......... .......... .......... 82% 279M 0s 2019-11-29T11:27:34.648 INFO:teuthology.orchestra.run.smithi069.stderr: 61050K .......... .......... .......... .......... .......... 83% 281M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61100K .......... .......... .......... .......... .......... 83% 275M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61150K .......... .......... .......... .......... .......... 83% 244M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61200K .......... .......... .......... .......... .......... 83% 287M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61250K .......... .......... .......... .......... .......... 83% 270M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61300K .......... .......... .......... .......... .......... 83% 282M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61350K .......... .......... .......... .......... .......... 83% 231M 0s 2019-11-29T11:27:34.649 INFO:teuthology.orchestra.run.smithi069.stderr: 61400K .......... .......... .......... .......... .......... 83% 289M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61450K .......... .......... .......... .......... .......... 83% 294M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61500K .......... .......... .......... .......... .......... 83% 295M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61550K .......... .......... .......... .......... .......... 83% 243M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61600K .......... .......... .......... .......... .......... 83% 279M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61650K .......... .......... .......... .......... .......... 83% 287M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61700K .......... .......... .......... .......... .......... 83% 286M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61750K .......... .......... .......... .......... .......... 84% 247M 0s 2019-11-29T11:27:34.650 INFO:teuthology.orchestra.run.smithi069.stderr: 61800K .......... .......... .......... .......... .......... 84% 278M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 61850K .......... .......... .......... .......... .......... 84% 267M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 61900K .......... .......... .......... .......... .......... 84% 265M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 61950K .......... .......... .......... .......... .......... 84% 239M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 62000K .......... .......... .......... .......... .......... 84% 286M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 62050K .......... .......... .......... .......... .......... 84% 266M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 62100K .......... .......... .......... .......... .......... 84% 276M 0s 2019-11-29T11:27:34.651 INFO:teuthology.orchestra.run.smithi069.stderr: 62150K .......... .......... .......... .......... .......... 84% 225M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62200K .......... .......... .......... .......... .......... 84% 281M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62250K .......... .......... .......... .......... .......... 84% 278M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62300K .......... .......... .......... .......... .......... 84% 281M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62350K .......... .......... .......... .......... .......... 84% 233M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62400K .......... .......... .......... .......... .......... 84% 266M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62450K .......... .......... .......... .......... .......... 84% 276M 0s 2019-11-29T11:27:34.652 INFO:teuthology.orchestra.run.smithi069.stderr: 62500K .......... .......... .......... .......... .......... 85% 276M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62550K .......... .......... .......... .......... .......... 85% 225M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62600K .......... .......... .......... .......... .......... 85% 266M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62650K .......... .......... .......... .......... .......... 85% 299M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62700K .......... .......... .......... .......... .......... 85% 261M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62750K .......... .......... .......... .......... .......... 85% 241M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62800K .......... .......... .......... .......... .......... 85% 281M 0s 2019-11-29T11:27:34.653 INFO:teuthology.orchestra.run.smithi069.stderr: 62850K .......... .......... .......... .......... .......... 85% 277M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 62900K .......... .......... .......... .......... .......... 85% 289M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 62950K .......... .......... .......... .......... .......... 85% 257M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 63000K .......... .......... .......... .......... .......... 85% 260M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 63050K .......... .......... .......... .......... .......... 85% 277M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 63100K .......... .......... .......... .......... .......... 85% 286M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 63150K .......... .......... .......... .......... .......... 85% 223M 0s 2019-11-29T11:27:34.654 INFO:teuthology.orchestra.run.smithi069.stderr: 63200K .......... .......... .......... .......... .......... 85% 275M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63250K .......... .......... .......... .......... .......... 86% 286M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63300K .......... .......... .......... .......... .......... 86% 273M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63350K .......... .......... .......... .......... .......... 86% 255M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63400K .......... .......... .......... .......... .......... 86% 285M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63450K .......... .......... .......... .......... .......... 86% 265M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63500K .......... .......... .......... .......... .......... 86% 280M 0s 2019-11-29T11:27:34.655 INFO:teuthology.orchestra.run.smithi069.stderr: 63550K .......... .......... .......... .......... .......... 86% 258M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63600K .......... .......... .......... .......... .......... 86% 273M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63650K .......... .......... .......... .......... .......... 86% 274M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63700K .......... .......... .......... .......... .......... 86% 289M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63750K .......... .......... .......... .......... .......... 86% 244M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63800K .......... .......... .......... .......... .......... 86% 267M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63850K .......... .......... .......... .......... .......... 86% 277M 0s 2019-11-29T11:27:34.656 INFO:teuthology.orchestra.run.smithi069.stderr: 63900K .......... .......... .......... .......... .......... 86% 275M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 63950K .......... .......... .......... .......... .......... 87% 215M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64000K .......... .......... .......... .......... .......... 87% 282M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64050K .......... .......... .......... .......... .......... 87% 267M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64100K .......... .......... .......... .......... .......... 87% 267M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64150K .......... .......... .......... .......... .......... 87% 255M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64200K .......... .......... .......... .......... .......... 87% 276M 0s 2019-11-29T11:27:34.657 INFO:teuthology.orchestra.run.smithi069.stderr: 64250K .......... .......... .......... .......... .......... 87% 283M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64300K .......... .......... .......... .......... .......... 87% 273M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64350K .......... .......... .......... .......... .......... 87% 234M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64400K .......... .......... .......... .......... .......... 87% 276M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64450K .......... .......... .......... .......... .......... 87% 276M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64500K .......... .......... .......... .......... .......... 87% 295M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64550K .......... .......... .......... .......... .......... 87% 237M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64600K .......... .......... .......... .......... .......... 87% 273M 0s 2019-11-29T11:27:34.658 INFO:teuthology.orchestra.run.smithi069.stderr: 64650K .......... .......... .......... .......... .......... 87% 288M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64700K .......... .......... .......... .......... .......... 88% 294M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64750K .......... .......... .......... .......... .......... 88% 224M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64800K .......... .......... .......... .......... .......... 88% 270M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64850K .......... .......... .......... .......... .......... 88% 273M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64900K .......... .......... .......... .......... .......... 88% 286M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 64950K .......... .......... .......... .......... .......... 88% 225M 0s 2019-11-29T11:27:34.659 INFO:teuthology.orchestra.run.smithi069.stderr: 65000K .......... .......... .......... .......... .......... 88% 289M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65050K .......... .......... .......... .......... .......... 88% 269M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65100K .......... .......... .......... .......... .......... 88% 278M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65150K .......... .......... .......... .......... .......... 88% 242M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65200K .......... .......... .......... .......... .......... 88% 268M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65250K .......... .......... .......... .......... .......... 88% 265M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65300K .......... .......... .......... .......... .......... 88% 289M 0s 2019-11-29T11:27:34.660 INFO:teuthology.orchestra.run.smithi069.stderr: 65350K .......... .......... .......... .......... .......... 88% 232M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65400K .......... .......... .......... .......... .......... 88% 274M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65450K .......... .......... .......... .......... .......... 89% 275M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65500K .......... .......... .......... .......... .......... 89% 293M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65550K .......... .......... .......... .......... .......... 89% 239M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65600K .......... .......... .......... .......... .......... 89% 260M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65650K .......... .......... .......... .......... .......... 89% 276M 0s 2019-11-29T11:27:34.661 INFO:teuthology.orchestra.run.smithi069.stderr: 65700K .......... .......... .......... .......... .......... 89% 245M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 65750K .......... .......... .......... .......... .......... 89% 224M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 65800K .......... .......... .......... .......... .......... 89% 277M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 65850K .......... .......... .......... .......... .......... 89% 262M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 65900K .......... .......... .......... .......... .......... 89% 248M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 65950K .......... .......... .......... .......... .......... 89% 224M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 66000K .......... .......... .......... .......... .......... 89% 271M 0s 2019-11-29T11:27:34.662 INFO:teuthology.orchestra.run.smithi069.stderr: 66050K .......... .......... .......... .......... .......... 89% 267M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66100K .......... .......... .......... .......... .......... 89% 270M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66150K .......... .......... .......... .......... .......... 89% 216M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66200K .......... .......... .......... .......... .......... 90% 259M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66250K .......... .......... .......... .......... .......... 90% 285M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66300K .......... .......... .......... .......... .......... 90% 288M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66350K .......... .......... .......... .......... .......... 90% 240M 0s 2019-11-29T11:27:34.663 INFO:teuthology.orchestra.run.smithi069.stderr: 66400K .......... .......... .......... .......... .......... 90% 283M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66450K .......... .......... .......... .......... .......... 90% 275M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66500K .......... .......... .......... .......... .......... 90% 271M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66550K .......... .......... .......... .......... .......... 90% 249M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66600K .......... .......... .......... .......... .......... 90% 272M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66650K .......... .......... .......... .......... .......... 90% 280M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66700K .......... .......... .......... .......... .......... 90% 264M 0s 2019-11-29T11:27:34.664 INFO:teuthology.orchestra.run.smithi069.stderr: 66750K .......... .......... .......... .......... .......... 90% 241M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 66800K .......... .......... .......... .......... .......... 90% 270M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 66850K .......... .......... .......... .......... .......... 90% 288M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 66900K .......... .......... .......... .......... .......... 91% 245M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 66950K .......... .......... .......... .......... .......... 91% 256M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 67000K .......... .......... .......... .......... .......... 91% 283M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 67050K .......... .......... .......... .......... .......... 91% 281M 0s 2019-11-29T11:27:34.665 INFO:teuthology.orchestra.run.smithi069.stderr: 67100K .......... .......... .......... .......... .......... 91% 293M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67150K .......... .......... .......... .......... .......... 91% 242M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67200K .......... .......... .......... .......... .......... 91% 273M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67250K .......... .......... .......... .......... .......... 91% 261M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67300K .......... .......... .......... .......... .......... 91% 278M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67350K .......... .......... .......... .......... .......... 91% 233M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67400K .......... .......... .......... .......... .......... 91% 289M 0s 2019-11-29T11:27:34.666 INFO:teuthology.orchestra.run.smithi069.stderr: 67450K .......... .......... .......... .......... .......... 91% 293M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67500K .......... .......... .......... .......... .......... 91% 295M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67550K .......... .......... .......... .......... .......... 91% 236M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67600K .......... .......... .......... .......... .......... 91% 292M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67650K .......... .......... .......... .......... .......... 92% 302M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67700K .......... .......... .......... .......... .......... 92% 267M 0s 2019-11-29T11:27:34.667 INFO:teuthology.orchestra.run.smithi069.stderr: 67750K .......... .......... .......... .......... .......... 92% 241M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 67800K .......... .......... .......... .......... .......... 92% 281M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 67850K .......... .......... .......... .......... .......... 92% 280M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 67900K .......... .......... .......... .......... .......... 92% 286M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 67950K .......... .......... .......... .......... .......... 92% 230M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 68000K .......... .......... .......... .......... .......... 92% 281M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 68050K .......... .......... .......... .......... .......... 92% 284M 0s 2019-11-29T11:27:34.668 INFO:teuthology.orchestra.run.smithi069.stderr: 68100K .......... .......... .......... .......... .......... 92% 284M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68150K .......... .......... .......... .......... .......... 92% 241M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68200K .......... .......... .......... .......... .......... 92% 279M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68250K .......... .......... .......... .......... .......... 92% 283M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68300K .......... .......... .......... .......... .......... 92% 282M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68350K .......... .......... .......... .......... .......... 92% 244M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68400K .......... .......... .......... .......... .......... 93% 263M 0s 2019-11-29T11:27:34.669 INFO:teuthology.orchestra.run.smithi069.stderr: 68450K .......... .......... .......... .......... .......... 93% 287M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68500K .......... .......... .......... .......... .......... 93% 270M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68550K .......... .......... .......... .......... .......... 93% 244M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68600K .......... .......... .......... .......... .......... 93% 277M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68650K .......... .......... .......... .......... .......... 93% 265M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68700K .......... .......... .......... .......... .......... 93% 274M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68750K .......... .......... .......... .......... .......... 93% 223M 0s 2019-11-29T11:27:34.670 INFO:teuthology.orchestra.run.smithi069.stderr: 68800K .......... .......... .......... .......... .......... 93% 267M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 68850K .......... .......... .......... .......... .......... 93% 272M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 68900K .......... .......... .......... .......... .......... 93% 274M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 68950K .......... .......... .......... .......... .......... 93% 242M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 69000K .......... .......... .......... .......... .......... 93% 290M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 69050K .......... .......... .......... .......... .......... 93% 269M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 69100K .......... .......... .......... .......... .......... 94% 270M 0s 2019-11-29T11:27:34.671 INFO:teuthology.orchestra.run.smithi069.stderr: 69150K .......... .......... .......... .......... .......... 94% 240M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69200K .......... .......... .......... .......... .......... 94% 286M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69250K .......... .......... .......... .......... .......... 94% 287M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69300K .......... .......... .......... .......... .......... 94% 268M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69350K .......... .......... .......... .......... .......... 94% 253M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69400K .......... .......... .......... .......... .......... 94% 277M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69450K .......... .......... .......... .......... .......... 94% 293M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69500K .......... .......... .......... .......... .......... 94% 267M 0s 2019-11-29T11:27:34.672 INFO:teuthology.orchestra.run.smithi069.stderr: 69550K .......... .......... .......... .......... .......... 94% 236M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69600K .......... .......... .......... .......... .......... 94% 302M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69650K .......... .......... .......... .......... .......... 94% 284M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69700K .......... .......... .......... .......... .......... 94% 277M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69750K .......... .......... .......... .......... .......... 94% 237M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69800K .......... .......... .......... .......... .......... 94% 288M 0s 2019-11-29T11:27:34.673 INFO:teuthology.orchestra.run.smithi069.stderr: 69850K .......... .......... .......... .......... .......... 95% 284M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 69900K .......... .......... .......... .......... .......... 95% 282M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 69950K .......... .......... .......... .......... .......... 95% 239M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 70000K .......... .......... .......... .......... .......... 95% 282M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 70050K .......... .......... .......... .......... .......... 95% 292M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 70100K .......... .......... .......... .......... .......... 95% 287M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 70150K .......... .......... .......... .......... .......... 95% 245M 0s 2019-11-29T11:27:34.674 INFO:teuthology.orchestra.run.smithi069.stderr: 70200K .......... .......... .......... .......... .......... 95% 298M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70250K .......... .......... .......... .......... .......... 95% 274M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70300K .......... .......... .......... .......... .......... 95% 269M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70350K .......... .......... .......... .......... .......... 95% 239M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70400K .......... .......... .......... .......... .......... 95% 267M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70450K .......... .......... .......... .......... .......... 95% 289M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70500K .......... .......... .......... .......... .......... 95% 273M 0s 2019-11-29T11:27:34.675 INFO:teuthology.orchestra.run.smithi069.stderr: 70550K .......... .......... .......... .......... .......... 95% 256M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70600K .......... .......... .......... .......... .......... 96% 268M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70650K .......... .......... .......... .......... .......... 96% 279M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70700K .......... .......... .......... .......... .......... 96% 284M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70750K .......... .......... .......... .......... .......... 96% 228M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70800K .......... .......... .......... .......... .......... 96% 313M 0s 2019-11-29T11:27:34.676 INFO:teuthology.orchestra.run.smithi069.stderr: 70850K .......... .......... .......... .......... .......... 96% 277M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 70900K .......... .......... .......... .......... .......... 96% 284M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 70950K .......... .......... .......... .......... .......... 96% 238M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 71000K .......... .......... .......... .......... .......... 96% 277M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 71050K .......... .......... .......... .......... .......... 96% 278M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 71100K .......... .......... .......... .......... .......... 96% 277M 0s 2019-11-29T11:27:34.677 INFO:teuthology.orchestra.run.smithi069.stderr: 71150K .......... .......... .......... .......... .......... 96% 226M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71200K .......... .......... .......... .......... .......... 96% 269M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71250K .......... .......... .......... .......... .......... 96% 284M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71300K .......... .......... .......... .......... .......... 96% 271M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71350K .......... .......... .......... .......... .......... 97% 256M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71400K .......... .......... .......... .......... .......... 97% 269M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71450K .......... .......... .......... .......... .......... 97% 287M 0s 2019-11-29T11:27:34.678 INFO:teuthology.orchestra.run.smithi069.stderr: 71500K .......... .......... .......... .......... .......... 97% 286M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71550K .......... .......... .......... .......... .......... 97% 235M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71600K .......... .......... .......... .......... .......... 97% 293M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71650K .......... .......... .......... .......... .......... 97% 289M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71700K .......... .......... .......... .......... .......... 97% 279M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71750K .......... .......... .......... .......... .......... 97% 260M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71800K .......... .......... .......... .......... .......... 97% 280M 0s 2019-11-29T11:27:34.679 INFO:teuthology.orchestra.run.smithi069.stderr: 71850K .......... .......... .......... .......... .......... 97% 266M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 71900K .......... .......... .......... .......... .......... 97% 286M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 71950K .......... .......... .......... .......... .......... 97% 227M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 72000K .......... .......... .......... .......... .......... 97% 286M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 72050K .......... .......... .......... .......... .......... 98% 270M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 72100K .......... .......... .......... .......... .......... 98% 284M 0s 2019-11-29T11:27:34.680 INFO:teuthology.orchestra.run.smithi069.stderr: 72150K .......... .......... .......... .......... .......... 98% 255M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72200K .......... .......... .......... .......... .......... 98% 262M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72250K .......... .......... .......... .......... .......... 98% 286M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72300K .......... .......... .......... .......... .......... 98% 288M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72350K .......... .......... .......... .......... .......... 98% 235M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72400K .......... .......... .......... .......... .......... 98% 277M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72450K .......... .......... .......... .......... .......... 98% 266M 0s 2019-11-29T11:27:34.681 INFO:teuthology.orchestra.run.smithi069.stderr: 72500K .......... .......... .......... .......... .......... 98% 266M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72550K .......... .......... .......... .......... .......... 98% 262M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72600K .......... .......... .......... .......... .......... 98% 269M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72650K .......... .......... .......... .......... .......... 98% 275M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72700K .......... .......... .......... .......... .......... 98% 289M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72750K .......... .......... .......... .......... .......... 98% 215M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72800K .......... .......... .......... .......... .......... 99% 286M 0s 2019-11-29T11:27:34.682 INFO:teuthology.orchestra.run.smithi069.stderr: 72850K .......... .......... .......... .......... .......... 99% 282M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 72900K .......... .......... .......... .......... .......... 99% 283M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 72950K .......... .......... .......... .......... .......... 99% 249M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 73000K .......... .......... .......... .......... .......... 99% 286M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 73050K .......... .......... .......... .......... .......... 99% 242M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 73100K .......... .......... .......... .......... .......... 99% 288M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 73150K .......... .......... .......... .......... .......... 99% 232M 0s 2019-11-29T11:27:34.683 INFO:teuthology.orchestra.run.smithi069.stderr: 73200K .......... .......... .......... .......... .......... 99% 290M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73250K .......... .......... .......... .......... .......... 99% 275M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73300K .......... .......... .......... .......... .......... 99% 293M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73350K .......... .......... .......... .......... .......... 99% 256M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73400K .......... .......... .......... .......... .......... 99% 85.1M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73450K .......... .......... .......... .......... .......... 99% 280M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73500K .......... .......... .......... .......... .......... 99% 281M 0s 2019-11-29T11:27:34.684 INFO:teuthology.orchestra.run.smithi069.stderr: 73550K ........ 100% 248M=0.3s 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29 11:27:33 (213 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stdout:Archive: gradle-4.7-bin.zip 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/ 2019-11-29T11:27:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-11-29T11:27:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-11-29T11:27:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-11-29T11:27:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-11-29T11:27:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-11-29T11:27:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-11-29T11:27:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-11-29T11:27:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-11-29T11:27:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-11-29T11:27:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-11-29T11:27:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-11-29T11:27:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-11-29T11:27:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-11-29T11:27:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-11-29T11:27:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-11-29T11:27:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-11-29T11:27:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-11-29T11:27:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-11-29T11:27:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-11-29T11:27:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-11-29T11:27:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-11-29T11:27:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-11-29T11:27:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-11-29T11:27:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-11-29T11:27:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-11-29T11:27:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-11-29T11:27:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-11-29T11:27:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-11-29T11:27:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-11-29T11:27:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-11-29T11:27:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-11-29T11:27:34.716 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-11-29T11:27:34.716 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-11-29T11:27:34.716 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-11-29T11:27:34.716 INFO:teuthology.orchestra.run.smithi069.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-11-29T11:27:34.716 INFO:teuthology.orchestra.run.smithi069.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout:Welcome to Gradle 4.7! 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout:Here are the highlights of this release: 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: - Incremental annotation processing 2019-11-29T11:27:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: - JDK 10 support 2019-11-29T11:27:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: - Grouped non-interactive console logs 2019-11-29T11:27:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: - Failed tests are re-run first for quicker feedback 2019-11-29T11:27:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.232 INFO:teuthology.orchestra.run.smithi069.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-11-29T11:27:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.593 INFO:teuthology.orchestra.run.smithi069.stdout:------------------------------------------------------------ 2019-11-29T11:27:35.593 INFO:teuthology.orchestra.run.smithi069.stdout:Gradle 4.7 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout:------------------------------------------------------------ 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout:Groovy: 2.4.12 2019-11-29T11:27:35.594 INFO:teuthology.orchestra.run.smithi069.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-11-29T11:27:35.595 INFO:teuthology.orchestra.run.smithi069.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-11-29T11:27:35.595 INFO:teuthology.orchestra.run.smithi069.stdout:OS: Linux 3.10.0-1062.4.3.el7.x86_64 amd64 2019-11-29T11:27:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:35.599 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-11-29T11:27:35.599 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-11-29T11:27:35.599 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi069.front.sepia.ceph.com 2019-11-29T11:27:35.600 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-11-29T11:27:35.600 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi163.front.sepia.ceph.com 2019-11-29T11:27:35.600 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-11-29T11:27:36.013 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-11-29T11:27:36.013 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-11-29T11:27:36.014 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'SNYBSVXPICRRRNKQKSUR', 's3dir': 'folder', 'endpoint': 'smithi069.front.sepia.ceph.com:80', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 80, 'access_secret': 'YZgBC+u8JaNJAkZGdLPqqWkq8qVlne35SX9Kh+C4Evrjc+7VdkkGHg==', 'host': 'smithi069.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': False, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-11-29T11:27:36.014 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-11-29T11:27:36.014 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'SNYBSVXPICRRRNKQKSUR', '--secret', 'YZgBC+u8JaNJAkZGdLPqqWkq8qVlne35SX9Kh+C4Evrjc+7VdkkGHg==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-11-29T11:27:36.014 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key SNYBSVXPICRRRNKQKSUR --secret YZgBC+u8JaNJAkZGdLPqqWkq8qVlne35SX9Kh+C4Evrjc+7VdkkGHg== --email tester.client.0_test@test.test --cluster ceph 2019-11-29T11:27:36.161 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.262+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.289+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.289+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.211 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.311+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.211 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.311+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.219 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.320+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.220 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.321+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.330+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:36.417 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.517+0000 7f1879969ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:27:36.417 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.517+0000 7f1879969ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:27:36.418 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.517+0000 7f1879969ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:27:36.418 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.517+0000 7f1879969ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:27:36.418 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.517+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.526+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.526+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.541+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:36.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.541+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.550+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:36.450 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.550+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.563+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:36.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.563+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.570+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.570+0000 7f1879969ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:27:36.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.570+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.578+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.578+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.588+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:36.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.588+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.503 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.603+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:36.503 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.603+0000 7f1879969ec0 20 zonegroup default 2019-11-29T11:27:36.503 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.603+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.608+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.609+0000 7f1879969ec0 10 Cannot find current period zone using local zone 2019-11-29T11:27:36.509 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.609+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.613+0000 7f1879969ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:36.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.613+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.620+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:36.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.620+0000 7f1879969ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:36.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.632+0000 7f1879969ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:36.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.632+0000 7f1879969ec0 20 zone default found 2019-11-29T11:27:36.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.632+0000 7f1879969ec0 20 started sync module instance, tier type = 2019-11-29T11:27:36.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.632+0000 7f1879969ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=0 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=1 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=2 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=3 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=4 2019-11-29T11:27:36.835 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=5 2019-11-29T11:27:36.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.935+0000 7f1879969ec0 20 add_watcher() i=6 2019-11-29T11:27:36.874 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.975+0000 7f1879969ec0 20 add_watcher() i=7 2019-11-29T11:27:36.875 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.975+0000 7f1879969ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:27:36.875 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.975+0000 7f18377de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:27:36.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:36.985+0000 7f1867397700 0 -- 172.21.15.69:0/3020439723 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55557cd3b480 msgr2=0x55557cd3b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:27:36.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:37.018+0000 7f1866b96700 0 -- 172.21.15.69:0/846937331 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7f17a80175e0 msgr2=0x7f17a8019ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:27:38.552 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.649+0000 7f17a5ffb700 20 reqs_thread_entry: start 2019-11-29T11:27:38.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.649+0000 7f17977fe700 20 reqs_thread_entry: start 2019-11-29T11:27:38.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.649+0000 7f1879969ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:27:38.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.650+0000 7f1795ffb700 20 reqs_thread_entry: start 2019-11-29T11:27:38.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.650+0000 7f1879969ec0 20 get_system_obj_state: rctx=0x7ffd62fba228 obj=default.rgw.meta:users.uid:tester.client.0 state=0x55557cdc0610 s->prefetch_data=0 2019-11-29T11:27:38.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.650+0000 7f1879969ec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-11-29T11:27:38.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.655+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-11-29T11:27:38.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.655+0000 7f1879969ec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-29T11:27:38.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.655+0000 7f1879969ec0 20 get_system_obj_state: rctx=0x7ffd62fba238 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x55557cdc23a0 s->prefetch_data=0 2019-11-29T11:27:38.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.655+0000 7f1879969ec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-11-29T11:27:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.660+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-11-29T11:27:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.660+0000 7f1879969ec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-29T11:27:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.660+0000 7f1879969ec0 20 get_system_obj_state: rctx=0x7ffd62fba238 obj=default.rgw.meta:users.keys:SNYBSVXPICRRRNKQKSUR state=0x55557cdc23a0 s->prefetch_data=0 2019-11-29T11:27:38.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.660+0000 7f1879969ec0 10 cache get: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR : miss 2019-11-29T11:27:38.575 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.673+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR info.flags=0x0 2019-11-29T11:27:38.575 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.673+0000 7f1879969ec0 10 adding default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR to cache LRU end 2019-11-29T11:27:38.575 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.673+0000 7f1879969ec0 20 get_system_obj_state: rctx=0x7ffd62fb9e48 obj=default.rgw.meta:users.keys:SNYBSVXPICRRRNKQKSUR state=0x55557cdc23a0 s->prefetch_data=0 2019-11-29T11:27:38.575 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.673+0000 7f1879969ec0 10 cache get: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR : type miss (requested=0x6, cached=0x0) 2019-11-29T11:27:38.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.686+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR info.flags=0x0 2019-11-29T11:27:38.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.686+0000 7f1879969ec0 10 moving default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR to cache LRU end 2019-11-29T11:27:38.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.686+0000 7f1879969ec0 20 get_system_obj_state: rctx=0x7ffd62fb9fc8 obj=default.rgw.meta:users.keys:SNYBSVXPICRRRNKQKSUR state=0x55557cdc23a0 s->prefetch_data=0 2019-11-29T11:27:38.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.686+0000 7f1879969ec0 10 cache get: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR : type miss (requested=0x6, cached=0x0) 2019-11-29T11:27:38.610 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.697+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR info.flags=0x0 2019-11-29T11:27:38.611 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.697+0000 7f1879969ec0 10 moving default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR to cache LRU end 2019-11-29T11:27:38.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.754+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-11-29T11:27:38.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.754+0000 7f1879969ec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-29T11:27:38.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.754+0000 7f1879969ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-11-29T11:27:38.676 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.776+0000 7f1847fff700 10 RGWWatcher::handle_notify() notify_id 571230650374 cookie 93825654778512 notifier 4505 bl.length()=632 2019-11-29T11:27:38.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.776+0000 7f1847fff700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-11-29T11:27:38.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.776+0000 7f1847fff700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-29T11:27:38.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.871+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-11-29T11:27:38.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.871+0000 7f1879969ec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-29T11:27:38.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.871+0000 7f1879969ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-11-29T11:27:38.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.880+0000 7f1847fff700 10 RGWWatcher::handle_notify() notify_id 571230650374 cookie 93825654746272 notifier 4505 bl.length()=200 2019-11-29T11:27:38.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.880+0000 7f1847fff700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-11-29T11:27:38.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.880+0000 7f1847fff700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-29T11:27:38.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.979+0000 7f1879969ec0 10 cache put: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR info.flags=0x7 2019-11-29T11:27:38.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.979+0000 7f1879969ec0 10 moving default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR to cache LRU end 2019-11-29T11:27:38.889 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.979+0000 7f1879969ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=189 2019-11-29T11:27:38.889 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.987+0000 7f1847fff700 10 RGWWatcher::handle_notify() notify_id 571230650375 cookie 93825654778512 notifier 4505 bl.length()=189 2019-11-29T11:27:38.889 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.987+0000 7f1847fff700 10 cache put: name=default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR info.flags=0x7 2019-11-29T11:27:38.889 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:38.987+0000 7f1847fff700 10 moving default.rgw.meta+users.keys+SNYBSVXPICRRRNKQKSUR to cache LRU end 2019-11-29T11:27:39.094 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T11:27:39.095 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "tester.client.0", 2019-11-29T11:27:39.095 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Ms. tester.client.0", 2019-11-29T11:27:39.095 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "tester.client.0_test@test.test", 2019-11-29T11:27:39.095 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T11:27:39.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T11:27:39.096 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T11:27:39.096 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T11:27:39.096 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T11:27:39.097 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "tester.client.0", 2019-11-29T11:27:39.097 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "SNYBSVXPICRRRNKQKSUR", 2019-11-29T11:27:39.097 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "YZgBC+u8JaNJAkZGdLPqqWkq8qVlne35SX9Kh+C4Evrjc+7VdkkGHg==" 2019-11-29T11:27:39.097 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T11:27:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T11:27:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T11:27:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T11:27:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T11:27:39.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T11:27:39.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T11:27:39.099 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T11:27:39.099 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T11:27:39.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:27:39.100 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:27:39.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:27:39.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:27:39.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:27:39.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:27:39.101 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T11:27:39.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T11:27:39.101 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T11:27:39.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T11:27:39.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T11:27:39.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T11:27:39.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T11:27:39.103 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T11:27:39.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T11:27:39.104 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T11:27:39.104 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T11:27:39.105 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:27:39.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.265+0000 7f1879969ec0 20 remove_watcher() i=0 2019-11-29T11:27:39.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.265+0000 7f1879969ec0 2 removed watcher, disabling cache 2019-11-29T11:27:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.328+0000 7f1879969ec0 20 remove_watcher() i=1 2019-11-29T11:27:39.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.373+0000 7f1879969ec0 20 remove_watcher() i=2 2019-11-29T11:27:39.336 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.437+0000 7f1879969ec0 20 remove_watcher() i=3 2019-11-29T11:27:39.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.498+0000 7f1879969ec0 20 remove_watcher() i=4 2019-11-29T11:27:39.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.549+0000 7f1879969ec0 20 remove_watcher() i=5 2019-11-29T11:27:39.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.614+0000 7f1879969ec0 20 remove_watcher() i=6 2019-11-29T11:27:39.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.680+0000 7f1879969ec0 20 remove_watcher() i=7 2019-11-29T11:27:39.593 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-11-29T11:27:39.674 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-11-29T11:27:39.675 INFO:teuthology.orchestra.run.smithi069:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-11-29T11:27:39.767 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:27:39.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:39.988+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:39.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.006+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:39.906 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.006+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:39.914 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.014+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:39.914 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.014+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.019+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.019+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:39.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.027+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:40.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.236+0000 7f23ae3d7ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:27:40.136 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.236+0000 7f23ae3d7ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:27:40.136 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.236+0000 7f23ae3d7ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:27:40.136 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.236+0000 7f23ae3d7ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:27:40.136 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.236+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.251+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:40.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.251+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.271+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.271+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.284+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:40.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.284+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.304+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:40.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.304+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.311+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:40.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.311+0000 7f23ae3d7ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:27:40.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.311+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.316+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:40.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.316+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.327+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:40.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.327+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.343+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:40.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.343+0000 7f23ae3d7ec0 20 zonegroup default 2019-11-29T11:27:40.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.344+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.248 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.349+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.349+0000 7f23ae3d7ec0 10 Cannot find current period zone using local zone 2019-11-29T11:27:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.349+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.354+0000 7f23ae3d7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.354+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.362+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:40.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.362+0000 7f23ae3d7ec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:40.279 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.380+0000 7f23ae3d7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:40.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.380+0000 7f23ae3d7ec0 20 zone default found 2019-11-29T11:27:40.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.380+0000 7f23ae3d7ec0 20 started sync module instance, tier type = 2019-11-29T11:27:40.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.380+0000 7f23ae3d7ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:27:40.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=0 2019-11-29T11:27:40.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=1 2019-11-29T11:27:40.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=2 2019-11-29T11:27:40.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=3 2019-11-29T11:27:40.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=4 2019-11-29T11:27:40.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=5 2019-11-29T11:27:40.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.807+0000 7f23ae3d7ec0 20 add_watcher() i=6 2019-11-29T11:27:40.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.836+0000 7f23ae3d7ec0 20 add_watcher() i=7 2019-11-29T11:27:40.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.836+0000 7f23ae3d7ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:27:40.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.837+0000 7f236bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:27:40.739 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.839+0000 7f239ae03700 0 -- 172.21.15.69:0/3709816063 >> v1:172.21.15.163:6789/0 conn(0x5555ffdd6350 legacy=0x5555ffdfc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-29T11:27:40.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:40.872+0000 7f239be05700 0 -- 172.21.15.69:0/1074087180 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7f22dc017580 msgr2=0x7f22dc019a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:27:42.282 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.382+0000 7f22da7fc700 20 reqs_thread_entry: start 2019-11-29T11:27:42.282 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.382+0000 7f22d3fff700 20 reqs_thread_entry: start 2019-11-29T11:27:42.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.382+0000 7f23ae3d7ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:27:42.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.383+0000 7f22d27fc700 20 reqs_thread_entry: start 2019-11-29T11:27:42.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.383+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:27:42.351 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.451+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:42.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.480+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:42.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:42.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.226', time=2019-11-29T13:27:08.961473+0000, chain.objs.size()=5 2019-11-29T11:27:42.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.1 2019-11-29T11:27:42.427 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.1_1 2019-11-29T11:27:42.427 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.2 2019-11-29T11:27:42.427 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.2_1 2019-11-29T11:27:42.427 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.526+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.3 2019-11-29T11:27:42.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.607+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:42.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.746+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:27:42.735 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.835+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.859+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_RAsocJ2QmxC-Vy_6ZK9qxmQIDAwtT6R', time=2019-11-29T13:27:05.237754+0000, chain.objs.size()=1 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.eRJ6zLkTbyRw_qoc9rpzVheO3o-Xnhx_1 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.236', time=2019-11-29T13:27:10.048720+0000, chain.objs.size()=5 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.1 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.1_1 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.2 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.2_1 2019-11-29T11:27:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:42.893+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.3 2019-11-29T11:27:42.953 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.052+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:43.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.176+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:27:43.121 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.221+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:43.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.238+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.274+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.274+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.84', time=2019-11-29T13:27:15.274952+0000, chain.objs.size()=3 2019-11-29T11:27:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.274+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.VIcPfUEKZ1etqSyxvfDU87kqVAt6p43_0 2019-11-29T11:27:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.274+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.VIcPfUEKZ1etqSyxvfDU87kqVAt6p43_1 2019-11-29T11:27:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.274+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.VIcPfUEKZ1etqSyxvfDU87kqVAt6p43_2 2019-11-29T11:27:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.403+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:43.409 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.509+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:27:43.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.580+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:43.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.608+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.637+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.637+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_a7GzwedOpjgMs7yRlr-HQ30lBxwhYyH', time=2019-11-29T13:27:06.206730+0000, chain.objs.size()=1 2019-11-29T11:27:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.637+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.ZGYCdHcQbzwDTvBMHXDjAcwNYdTmWaS_1 2019-11-29T11:27:43.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.724+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:43.946+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:27:43.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.011+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:43.934 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.034+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:43.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.047+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:43.947 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.047+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_kNpDVGEqgNV_oS6yCobqP1MeGNaiSYA', time=2019-11-29T13:27:07.119458+0000, chain.objs.size()=2 2019-11-29T11:27:43.947 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.047+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NxxyKSCOyQHkpF4eh_HkGnj5bvyNu7U_0 2019-11-29T11:27:43.947 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.047+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NxxyKSCOyQHkpF4eh_HkGnj5bvyNu7U_1 2019-11-29T11:27:44.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.207+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:44.199 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.299+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:27:44.239 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.339+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.355+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.114', time=2019-11-29T13:27:06.038302+0000, chain.objs.size()=2 2019-11-29T11:27:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.4fkEnxJAx5jtHjai7SaWr-KP7TSHWQZ_0 2019-11-29T11:27:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.4fkEnxJAx5jtHjai7SaWr-KP7TSHWQZ_1 2019-11-29T11:27:44.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.116', time=2019-11-29T13:27:06.622446+0000, chain.objs.size()=1 2019-11-29T11:27:44.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.367+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.-caNIpJwDCOG-u8MphyUZDByO-I28RA_1 2019-11-29T11:27:44.346 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.446+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:44.422 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.522+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:27:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.607+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:44.534 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.635+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:44.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:44.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_NMnAehu1Ia4Jl7ow7xskmNN6r11Bau5', time=2019-11-29T13:27:09.362300+0000, chain.objs.size()=2 2019-11-29T11:27:44.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.6ByAspQ7c7bY5IYPfUKT3UohBuHW3Lw_1 2019-11-29T11:27:44.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.6ByAspQ7c7bY5IYPfUKT3UohBuHW3Lw_2 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.234', time=2019-11-29T13:27:09.623588+0000, chain.objs.size()=5 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.1 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.1_1 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.679+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.2 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.2_1 2019-11-29T11:27:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-LUKEWARM.2~ZMxPQojcL-VYdv_7dJ2gAGwqmvZuYtE.3 2019-11-29T11:27:44.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:44.915+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:44.948 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.048+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:27:44.994 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.094+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.113+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-29T11:27:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_D3mNovQSeu7DrM6GxP7O_Rpe8X0RKDF', time=2019-11-29T13:27:06.755856+0000, chain.objs.size()=2 2019-11-29T11:27:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.SyypWyMiPl_iVtNqN1l7T75y3869Zbc_0 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.SyypWyMiPl_iVtNqN1l7T75y3869Zbc_1 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.246', time=2019-11-29T13:27:11.040325+0000, chain.objs.size()=3 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.1 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.2 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.3 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.197', time=2019-11-29T13:27:13.126093+0000, chain.objs.size()=5 2019-11-29T11:27:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-FROZEN.2~0nJXFGt0yseQacbWJS0K1ParpmYtcuf.1 2019-11-29T11:27:45.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__shadow_foo-FROZEN.2~0nJXFGt0yseQacbWJS0K1ParpmYtcuf.1_1 2019-11-29T11:27:45.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-FROZEN.2~0nJXFGt0yseQacbWJS0K1ParpmYtcuf.2 2019-11-29T11:27:45.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__shadow_foo-FROZEN.2~0nJXFGt0yseQacbWJS0K1ParpmYtcuf.2_1 2019-11-29T11:27:45.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.146+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-FROZEN.2~0nJXFGt0yseQacbWJS0K1ParpmYtcuf.3 2019-11-29T11:27:45.198 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.298+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:45.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.404+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:27:45.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.481+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.511+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:45.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.528+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.581+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:27:45.548 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.649+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.676+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.696+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.747+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:27:45.715 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.816+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.850+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.55', time=2019-11-29T13:27:02.007056+0000, chain.objs.size()=7 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__multipart_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.1 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__shadow_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.1_1 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__multipart_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.2 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__shadow_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.2_1 2019-11-29T11:27:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__multipart_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.3 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__shadow_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.3_1 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.4__multipart_foo.2~2knmaPwOf6Y6BCZSXk-5zW_dczdizbi.4 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.109', time=2019-11-29T13:27:04.924982+0000, chain.objs.size()=2 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NPQ9Qgk-ni9FAB-cFBOXEnYW5NEypPV_0 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NPQ9Qgk-ni9FAB-cFBOXEnYW5NEypPV_1 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.112', time=2019-11-29T13:27:05.529225+0000, chain.objs.size()=2 2019-11-29T11:27:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.4fkEnxJAx5jtHjai7SaWr-KP7TSHWQZ_0 2019-11-29T11:27:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:45.872+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.4fkEnxJAx5jtHjai7SaWr-KP7TSHWQZ_1 2019-11-29T11:27:46.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.211+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:46.181 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.282+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:27:46.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.329+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.347+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:46.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.356+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.291 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.393+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:27:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.455+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.476+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.489+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.534+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:27:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.600+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.519 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.620+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:46.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.629+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.675+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:27:46.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.756+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:46.687 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.788+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:46.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:46.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_h4xBYONgI1EnW9zf2RmD2Gw5oaCO173', time=2019-11-29T13:27:10.252020+0000, chain.objs.size()=2 2019-11-29T11:27:46.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.Rh2Kl4hJTCjyONFTU7KPSnyNBx_aO0H_1 2019-11-29T11:27:46.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.Rh2Kl4hJTCjyONFTU7KPSnyNBx_aO0H_2 2019-11-29T11:27:46.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_20Hf_J8N7JM4ZX_GkkfKpMZPT3ce0r6', time=2019-11-29T13:27:10.594294+0000, chain.objs.size()=1 2019-11-29T11:27:46.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.812+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.kVpvdygo4_4XKLBR5fuOEzkayqUBOyE_0 2019-11-29T11:27:46.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:46.947+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:46.917 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.019+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:27:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.082+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.104+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:47.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.113+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:47.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.113+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_yuFVdKMmo24cQvcOzY4tGOSZefk8Nt3', time=2019-11-29T13:27:05.818321+0000, chain.objs.size()=2 2019-11-29T11:27:47.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.113+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.47AE_-xSNhdjwaFeDoeku_RfoIx0Uig_0 2019-11-29T11:27:47.013 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.114+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.47AE_-xSNhdjwaFeDoeku_RfoIx0Uig_1 2019-11-29T11:27:47.099 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.200+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:47.181 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.282+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:27:47.272 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.374+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:47.324 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.426+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:47.370 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:47.370 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.119', time=2019-11-29T13:27:07.302372+0000, chain.objs.size()=1 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.-caNIpJwDCOG-u8MphyUZDByO-I28RA_1 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.225', time=2019-11-29T13:27:08.638389+0000, chain.objs.size()=5 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.1 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.1_1 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.2 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.471+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.2_1 2019-11-29T11:27:47.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.472+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-FROZEN.2~wYyOTDGTP2_lHqw6QWm10Os3uFBoIy9.3 2019-11-29T11:27:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.684+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.835+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:27:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.904+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.931+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:47.946+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:47.899 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.000+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:27:47.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.063+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:47.987 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.089+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:48.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.128+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:48.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.128+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.85', time=2019-11-29T13:27:15.512216+0000, chain.objs.size()=3 2019-11-29T11:27:48.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.128+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.hO8lFJRnC4ZC5KyLG3E-_uarcbvib4T_0 2019-11-29T11:27:48.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.129+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.hO8lFJRnC4ZC5KyLG3E-_uarcbvib4T_1 2019-11-29T11:27:48.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.129+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.6__shadow_.hO8lFJRnC4ZC5KyLG3E-_uarcbvib4T_2 2019-11-29T11:27:48.141 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.243+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:48.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.424+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:27:48.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.503+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:48.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.534+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:48.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.544+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.581+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:27:48.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.625+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.641+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2019-11-29T11:27:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_JQ9Xud5dM7onFGBeHrL-GagcDHgu3Ac', time=2019-11-29T13:27:05.097916+0000, chain.objs.size()=2 2019-11-29T11:27:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.X0NcYzYg5-ypsgy0u3nS90QDCka3owm_0 2019-11-29T11:27:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.X0NcYzYg5-ypsgy0u3nS90QDCka3owm_1 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_TUIiX-8YDsQR5BJAj632NEga4RF6cbo', time=2019-11-29T13:27:09.148211+0000, chain.objs.size()=3 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.7UKCjoUWKh8-G2aEA4XnoR5WgXlpM46_0 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.7UKCjoUWKh8-G2aEA4XnoR5WgXlpM46_1 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.7UKCjoUWKh8-G2aEA4XnoR5WgXlpM46_2 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_LnPuUqH2G0PysXNb9LtNsk_ykldY0BB', time=2019-11-29T13:27:09.926983+0000, chain.objs.size()=3 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.ZHIbu4-xOyrmRmw5RuK_xmrqnWPi3J3_0 2019-11-29T11:27:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.ZHIbu4-xOyrmRmw5RuK_xmrqnWPi3J3_1 2019-11-29T11:27:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.ZHIbu4-xOyrmRmw5RuK_xmrqnWPi3J3_2 2019-11-29T11:27:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.211', time=2019-11-29T13:27:13.623958+0000, chain.objs.size()=3 2019-11-29T11:27:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-STANDARD.2~qlxwqTgzZ4ZpzULcjdSjTbToIctXpMT.1 2019-11-29T11:27:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-STANDARD.2~qlxwqTgzZ4ZpzULcjdSjTbToIctXpMT.2 2019-11-29T11:27:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.680+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-STANDARD.2~qlxwqTgzZ4ZpzULcjdSjTbToIctXpMT.3 2019-11-29T11:27:48.785 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.887+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:48.997+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:27:48.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.037+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:48.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.053+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:48.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.063+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:48.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.063+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='_0QfXa_ct3wTKPgAOmQ8bByN2iGPmepM', time=2019-11-29T13:27:10.909349+0000, chain.objs.size()=1 2019-11-29T11:27:48.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.063+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__shadow_.AsNaoQhgtSWt_mE9xjhqOIlSEUNSwu0_0 2019-11-29T11:27:49.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.127+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:49.212 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.313+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:27:49.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.391+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.430+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:49.358 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:49.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.71', time=2019-11-29T13:27:03.408223+0000, chain.objs.size()=4 2019-11-29T11:27:49.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5__multipart_foo.2~izKRpjY9uWLyyVgRZfPVwDpaMboQdeD.1 2019-11-29T11:27:49.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5__multipart_foo.2~izKRpjY9uWLyyVgRZfPVwDpaMboQdeD.2 2019-11-29T11:27:49.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5__multipart_foo.2~izKRpjY9uWLyyVgRZfPVwDpaMboQdeD.3 2019-11-29T11:27:49.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.460+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.5__multipart_foo.2~izKRpjY9uWLyyVgRZfPVwDpaMboQdeD.4 2019-11-29T11:27:49.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.675+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.781+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:27:49.722 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.824+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:49.738 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.840+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:49.749 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.851+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:49.749 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.851+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.243', time=2019-11-29T13:27:10.420129+0000, chain.objs.size()=3 2019-11-29T11:27:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.851+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.1 2019-11-29T11:27:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.851+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.2 2019-11-29T11:27:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:49.851+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.9__multipart_foo-STANDARD.2~d5MgjzLTY1KtQtbBiOOiGSM68JdaRNS.3 2019-11-29T11:27:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.011+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:49.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.077+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:27:50.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.121+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.036 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.138+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:50.045 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.147+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.182+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:27:50.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.239+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.273+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:50.179 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.281+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.318+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:27:50.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.356+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.372+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:50.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.380+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.418+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:27:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.469+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.488+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.496+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.545+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:27:50.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.609+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:50.533 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.635+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:50.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:50.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.20', time=2019-11-29T13:26:58.884623+0000, chain.objs.size()=7 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__multipart_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.1 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__shadow_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.1_1 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__multipart_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.2 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__shadow_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.2_1 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__multipart_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.3 2019-11-29T11:27:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__shadow_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.3_1 2019-11-29T11:27:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.650+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.2__multipart_foo.2~qBMkmguxoXKU7UNz7MKuRYJs77U-2XU.4 2019-11-29T11:27:50.875 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:50.977+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:50.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.065+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:27:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.108+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.128+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.138+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:51.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.169+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:27:51.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.208+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:51.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.231+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:27:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.40', time=2019-11-29T13:27:00.216367+0000, chain.objs.size()=7 2019-11-29T11:27:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__multipart_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.1 2019-11-29T11:27:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__shadow_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.1_1 2019-11-29T11:27:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__multipart_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.2 2019-11-29T11:27:51.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__shadow_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.2_1 2019-11-29T11:27:51.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.246+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__multipart_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.3 2019-11-29T11:27:51.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__shadow_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.3_1 2019-11-29T11:27:51.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.3__multipart_foo.2~RseUgXVzaRCz5czekNKGNE_DLSAlXZt.4 2019-11-29T11:27:51.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.204', time=2019-11-29T13:27:13.399225+0000, chain.objs.size()=5 2019-11-29T11:27:51.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-LUKEWARM.2~51NPw-N4DhVdhddLWnT-76nHrsH9aOC.1 2019-11-29T11:27:51.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__shadow_foo-LUKEWARM.2~51NPw-N4DhVdhddLWnT-76nHrsH9aOC.1_1 2019-11-29T11:27:51.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-LUKEWARM.2~51NPw-N4DhVdhddLWnT-76nHrsH9aOC.2 2019-11-29T11:27:51.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__shadow_foo-LUKEWARM.2~51NPw-N4DhVdhddLWnT-76nHrsH9aOC.2_1 2019-11-29T11:27:51.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.247+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:42bb32be-1e05-491f-815b-a606bdf24d77.4355.8__multipart_foo-LUKEWARM.2~51NPw-N4DhVdhddLWnT-76nHrsH9aOC.3 2019-11-29T11:27:51.472 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.574+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:51.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.657+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:27:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.701+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.718+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:51.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.731+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:27:51.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.731+0000 7f23ae3d7ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.108', time=2019-11-29T13:27:04.679526+0000, chain.objs.size()=2 2019-11-29T11:27:51.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.731+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NPQ9Qgk-ni9FAB-cFBOXEnYW5NEypPV_0 2019-11-29T11:27:51.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.731+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:42bb32be-1e05-491f-815b-a606bdf24d77.4355.7__shadow_.NPQ9Qgk-ni9FAB-cFBOXEnYW5NEypPV_1 2019-11-29T11:27:51.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.809+0000 7f23ae3d7ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:27:51.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.940+0000 7f23ae3d7ec0 20 remove_watcher() i=0 2019-11-29T11:27:51.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.940+0000 7f23ae3d7ec0 2 removed watcher, disabling cache 2019-11-29T11:27:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:51.979+0000 7f23ae3d7ec0 20 remove_watcher() i=1 2019-11-29T11:27:51.910 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.011+0000 7f23ae3d7ec0 20 remove_watcher() i=2 2019-11-29T11:27:51.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.089+0000 7f23ae3d7ec0 20 remove_watcher() i=3 2019-11-29T11:27:52.058 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.159+0000 7f23ae3d7ec0 20 remove_watcher() i=4 2019-11-29T11:27:52.115 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.216+0000 7f23ae3d7ec0 20 remove_watcher() i=5 2019-11-29T11:27:52.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.271+0000 7f23ae3d7ec0 20 remove_watcher() i=6 2019-11-29T11:27:52.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.326+0000 7f23ae3d7ec0 20 remove_watcher() i=7 2019-11-29T11:27:52.235 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:27:52.364 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.465+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.486+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.486+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.496+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.496+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.505+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.505+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.514+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:52.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.674+0000 7fe65182dec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:27:52.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.674+0000 7fe65182dec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:27:52.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.674+0000 7fe65182dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:27:52.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.674+0000 7fe65182dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:27:52.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.674+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.582 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.683+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:52.582 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.683+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.692+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:52.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.692+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.703+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:52.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.703+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.609 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.710+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:52.609 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.710+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.613 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.715+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.715+0000 7fe65182dec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:27:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.715+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.618 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.719+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.618 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.719+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.728+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:52.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.728+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.743+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:27:52.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.743+0000 7fe65182dec0 20 zonegroup default 2019-11-29T11:27:52.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.743+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.749+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.749+0000 7fe65182dec0 10 Cannot find current period zone using local zone 2019-11-29T11:27:52.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.749+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.757+0000 7fe65182dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:27:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.757+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.763+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:27:52.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.763+0000 7fe65182dec0 20 rados->read ofs=0 len=0 2019-11-29T11:27:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.784+0000 7fe65182dec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:27:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.784+0000 7fe65182dec0 20 zone default found 2019-11-29T11:27:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.784+0000 7fe65182dec0 20 started sync module instance, tier type = 2019-11-29T11:27:52.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:52.784+0000 7fe65182dec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:27:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=0 2019-11-29T11:27:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=1 2019-11-29T11:27:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=2 2019-11-29T11:27:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=3 2019-11-29T11:27:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=4 2019-11-29T11:27:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=5 2019-11-29T11:27:53.051 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.150+0000 7fe65182dec0 20 add_watcher() i=6 2019-11-29T11:27:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.208+0000 7fe65182dec0 20 add_watcher() i=7 2019-11-29T11:27:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.208+0000 7fe65182dec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:27:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.209+0000 7fe60f7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:27:53.113 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.214+0000 7fe637fff700 0 -- 172.21.15.69:0/2639033690 >> v1:172.21.15.163:6789/0 conn(0x55c9996739e0 legacy=0x55c999653640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:27:53.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:53.253+0000 7fe637fff700 0 -- 172.21.15.69:0/128729223 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x55c9996a0f50 msgr2=0x55c9996a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:27:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.033+0000 7fe57dffb700 20 reqs_thread_entry: start 2019-11-29T11:27:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.033+0000 7fe5777fe700 20 reqs_thread_entry: start 2019-11-29T11:27:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.034+0000 7fe65182dec0 20 init_complete bucket index max shards: 0 2019-11-29T11:27:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.034+0000 7fe575ffb700 20 reqs_thread_entry: start 2019-11-29T11:27:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.034+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:27:54.969 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.070+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:54.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.086+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:54.994 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.095+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.129+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:27:55.063 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.165+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.181+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.190+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.120 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.221+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:27:55.179 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.281+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.300+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.312+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.355+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:27:55.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.429+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.457+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.365 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.467+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.524+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:27:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.589+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.613+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.628+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.666+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:27:55.632 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.733+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.656 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.757+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.665 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.767+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.811+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:27:55.749 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.850+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.765 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.867+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.877+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.908+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:27:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.954+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.870 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.971+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:55.893 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:55.994+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:55.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.025+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:27:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.106+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.130+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.142+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.180+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:27:56.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.229+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.245+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.155 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.256+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.296+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:27:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.356+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.272 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.373+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.381+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.327 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.428+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:27:56.410 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.511+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.445 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.546+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.462 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.564+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.629+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:27:56.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.678+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.695+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.704+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.731+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:27:56.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.802+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.831+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.744 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.845+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.899+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:27:56.870 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:56.972+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.901 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.002+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:56.920 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.021+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:56.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.082+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:27:57.032 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.134+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.156+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.169+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.213+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:27:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.279+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.200 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.303+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.318+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.370+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:27:57.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.422+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.445+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.459+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.499+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:27:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.564+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.592+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.604+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.555 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.658+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:27:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.750+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.780+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.691 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.793+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.744 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.846+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:27:57.847 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.949+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.988+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:57.897 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:57.999+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.035+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:27:57.972 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.075+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.107+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.125+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.153+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:27:58.086 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.189+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.203+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.109 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.212+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.237+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:27:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.274+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.289+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.297+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.325+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:27:58.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.397+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.431+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.443+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.383 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.486+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:27:58.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.539+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.556+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.464 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.567+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.613+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:27:58.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.667+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.692+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.702+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.644 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.746+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:27:58.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.820+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.848+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.863+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.813 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.916+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:27:58.866 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.969+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:58.991+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:58.898 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.001+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:58.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.029+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:27:58.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.085+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.009 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.102+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:59.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.113+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.156+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:27:59.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.215+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.237+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:59.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.246+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.294+0000 7fe65182dec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:27:59.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.359+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.287 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.390+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:27:59.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.403+0000 7fe65182dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:27:59.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.524+0000 7fe65182dec0 20 remove_watcher() i=0 2019-11-29T11:27:59.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.524+0000 7fe65182dec0 2 removed watcher, disabling cache 2019-11-29T11:27:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.576+0000 7fe65182dec0 20 remove_watcher() i=1 2019-11-29T11:27:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.617+0000 7fe65182dec0 20 remove_watcher() i=2 2019-11-29T11:27:59.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.674+0000 7fe65182dec0 20 remove_watcher() i=3 2019-11-29T11:27:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.735+0000 7fe65182dec0 20 remove_watcher() i=4 2019-11-29T11:27:59.683 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.786+0000 7fe65182dec0 20 remove_watcher() i=5 2019-11-29T11:27:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.837+0000 7fe65182dec0 20 remove_watcher() i=6 2019-11-29T11:27:59.777 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:27:59.879+0000 7fe65182dec0 20 remove_watcher() i=7 2019-11-29T11:27:59.787 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-11-29T11:28:00.415 INFO:teuthology.orchestra.run.smithi069.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-11-29T11:28:03.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-11-29T11:28:03.913 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-11-29T11:28:05.115 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :clean UP-TO-DATE 2019-11-29T11:28:05.213 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-11-29T11:28:05.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-11-29T11:28:05.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-11-29T11:28:05.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-11-29T11:28:05.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-11-29T11:28:05.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-11-29T11:28:05.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-11-29T11:28:05.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-11-29T11:28:05.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-11-29T11:28:05.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-11-29T11:28:05.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-11-29T11:28:05.516 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-11-29T11:28:05.516 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-11-29T11:28:05.516 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-11-29T11:28:05.516 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-11-29T11:28:05.517 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-11-29T11:28:05.517 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-11-29T11:28:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-11-29T11:28:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-11-29T11:28:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-11-29T11:28:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-11-29T11:28:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-11-29T11:28:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-11-29T11:28:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-11-29T11:28:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-11-29T11:28:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-11-29T11:28:05.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-11-29T11:28:05.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-11-29T11:28:05.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-11-29T11:28:05.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-11-29T11:28:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-11-29T11:28:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-11-29T11:28:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-11-29T11:28:05.721 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-11-29T11:28:05.722 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-11-29T11:28:05.723 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-11-29T11:28:05.723 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-11-29T11:28:05.723 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-11-29T11:28:05.723 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-11-29T11:28:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-11-29T11:28:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-11-29T11:28:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-11-29T11:28:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-11-29T11:28:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-11-29T11:28:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-11-29T11:28:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-11-29T11:28:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-11-29T11:28:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-11-29T11:28:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-11-29T11:28:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-11-29T11:28:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-11-29T11:28:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-11-29T11:28:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-11-29T11:28:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-11-29T11:28:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-11-29T11:28:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-11-29T11:28:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-11-29T11:28:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-11-29T11:28:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-11-29T11:28:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-11-29T11:28:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-11-29T11:28:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-11-29T11:28:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-11-29T11:28:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-11-29T11:28:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-11-29T11:28:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-11-29T11:28:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-11-29T11:28:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-11-29T11:28:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-11-29T11:28:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-11-29T11:28:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-11-29T11:28:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-11-29T11:28:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-11-29T11:28:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-11-29T11:28:06.213 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-11-29T11:28:06.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-11-29T11:28:06.215 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-11-29T11:28:06.215 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-11-29T11:28:06.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-11-29T11:28:06.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-11-29T11:28:06.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-11-29T11:28:06.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-11-29T11:28:06.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-11-29T11:28:06.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-11-29T11:28:06.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-11-29T11:28:06.316 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-11-29T11:28:06.316 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-11-29T11:28:06.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-11-29T11:28:06.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-11-29T11:28:06.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-11-29T11:28:06.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-11-29T11:28:06.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-11-29T11:28:06.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-11-29T11:28:06.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-11-29T11:28:06.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-11-29T11:28:06.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-11-29T11:28:06.417 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-11-29T11:28:06.417 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-11-29T11:28:06.417 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-11-29T11:28:06.417 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-11-29T11:28:06.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-11-29T11:28:06.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-11-29T11:28:06.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-11-29T11:28:06.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-11-29T11:28:06.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-11-29T11:28:06.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-11-29T11:28:06.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-11-29T11:28:06.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-11-29T11:28:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-11-29T11:28:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-11-29T11:28:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-11-29T11:28:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-11-29T11:28:06.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-11-29T11:28:06.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-11-29T11:28:06.713 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-11-29T11:28:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-11-29T11:28:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-11-29T11:28:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-11-29T11:28:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-11-29T11:28:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-11-29T11:28:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-11-29T11:28:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-11-29T11:28:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-11-29T11:28:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-11-29T11:28:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-11-29T11:28:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-11-29T11:28:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-11-29T11:28:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-11-29T11:28:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-11-29T11:28:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-11-29T11:28:07.513 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-11-29T11:28:07.513 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-11-29T11:28:07.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-11-29T11:28:07.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-11-29T11:28:07.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-11-29T11:28:07.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-11-29T11:28:07.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-11-29T11:28:07.618 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-11-29T11:28:07.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-11-29T11:28:07.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-11-29T11:28:07.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-11-29T11:28:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-11-29T11:28:07.718 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-11-29T11:28:07.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-11-29T11:28:07.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-11-29T11:28:07.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-11-29T11:28:07.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-11-29T11:28:07.719 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-11-29T11:28:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-11-29T11:28:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-11-29T11:28:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-11-29T11:28:07.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-11-29T11:28:07.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-11-29T11:28:07.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-11-29T11:28:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-11-29T11:28:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-11-29T11:28:07.818 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-11-29T11:28:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-11-29T11:28:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-11-29T11:28:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-11-29T11:28:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-11-29T11:28:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-11-29T11:28:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-11-29T11:28:07.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-11-29T11:28:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-11-29T11:28:07.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-11-29T11:28:07.917 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-11-29T11:28:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-11-29T11:28:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-11-29T11:28:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-11-29T11:28:07.918 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-11-29T11:28:08.015 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-11-29T11:28:08.015 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-11-29T11:28:08.015 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-11-29T11:28:08.016 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-11-29T11:28:08.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-11-29T11:28:08.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-11-29T11:28:08.017 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-11-29T11:28:08.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-11-29T11:28:08.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-11-29T11:28:08.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-11-29T11:28:08.114 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-11-29T11:28:08.115 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-11-29T11:28:08.116 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-11-29T11:28:08.213 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-11-29T11:28:08.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-11-29T11:28:08.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-11-29T11:28:08.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-11-29T11:28:08.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-11-29T11:28:08.214 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-11-29T11:28:08.215 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-11-29T11:28:08.215 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-11-29T11:28:08.216 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-11-29T11:28:08.216 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-11-29T11:28:08.216 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-11-29T11:28:08.216 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-11-29T11:28:08.216 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-11-29T11:28:08.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-11-29T11:28:08.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-11-29T11:28:08.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-11-29T11:28:08.314 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-11-29T11:28:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-11-29T11:28:08.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-11-29T11:28:08.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-11-29T11:28:08.414 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-11-29T11:28:08.415 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-11-29T11:28:08.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-11-29T11:28:08.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-11-29T11:28:08.416 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-11-29T11:28:08.513 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-11-29T11:28:08.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-11-29T11:28:08.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-11-29T11:28:08.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-11-29T11:28:08.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-11-29T11:28:08.514 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-11-29T11:28:08.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-11-29T11:28:08.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-11-29T11:28:08.515 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-11-29T11:28:08.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-11-29T11:28:08.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-11-29T11:28:08.614 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-11-29T11:28:08.615 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-11-29T11:28:08.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-11-29T11:28:08.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-11-29T11:28:08.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-11-29T11:28:08.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-11-29T11:28:08.616 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-11-29T11:28:08.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-11-29T11:28:08.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-11-29T11:28:08.617 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-11-29T11:28:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-11-29T11:28:08.713 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-11-29T11:28:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-11-29T11:28:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-11-29T11:28:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-11-29T11:28:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-11-29T11:28:08.714 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-11-29T11:28:08.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-11-29T11:28:08.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-11-29T11:28:08.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-11-29T11:28:08.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-11-29T11:28:08.715 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-11-29T11:28:08.716 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-11-29T11:28:08.717 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-11-29T11:28:08.814 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-11-29T11:28:08.814 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-11-29T11:28:08.814 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-11-29T11:28:08.815 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-11-29T11:28:08.816 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-11-29T11:28:08.817 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-11-29T11:28:08.913 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-11-29T11:28:08.914 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-11-29T11:28:08.915 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-11-29T11:28:08.916 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-11-29T11:28:09.013 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-11-29T11:28:09.013 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-11-29T11:28:09.013 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-11-29T11:28:09.014 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-11-29T11:28:09.015 INFO:teuthology.orchestra.run.smithi069.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-11-29T11:28:17.712 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileJava 2019-11-29T11:28:17.712 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processResources 2019-11-29T11:28:17.712 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :classes 2019-11-29T11:28:18.112 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:28:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileTestJava 2019-11-29T11:28:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-29T11:28:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-29T11:28:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:28:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processTestResources NO-SOURCE 2019-11-29T11:28:18.114 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :testClasses 2019-11-29T11:28:24.911 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:28:24.911 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :test 2019-11-29T11:28:24.911 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:28:24.912 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-11-29T11:29:00.007 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:00.007 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-11-29T11:29:06.206 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:06.206 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-11-29T11:29:10.014 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:10.014 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-11-29T11:29:13.717 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:16:26.581 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.752 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:16:26.614 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.753 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:16:26.615 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.763 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:16:26.561 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.777 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:16:26.568 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.782 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:16:26.644 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.922 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:16:26.787 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.946 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:16:26.809 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:13.984 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:16:26.778 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:29:15.905 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:15.905 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-11-29T11:29:40.602 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:40.602 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-11-29T11:29:46.806 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:46.807 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-11-29T11:29:47.201 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:47.202 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-11-29T11:29:47.901 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:47.902 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-11-29T11:29:51.301 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:51.301 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-11-29T11:29:58.800 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:29:58.800 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-11-29T11:30:15.491 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:17:28.361 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.509 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:17:28.378 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.519 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:17:28.316 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.521 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:17:28.390 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.534 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:17:28.404 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.548 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:17:28.419 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.571 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:17:28.371 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.586 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:17:28.457 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:15.654 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:17:28.450 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:30:20.598 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:30:20.598 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-11-29T11:30:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:30:36.296 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-11-29T11:33:59.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:33:59.180 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-11-29T11:34:12.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:12.680 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-11-29T11:34:15.779 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:15.779 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-11-29T11:34:21.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:21.580 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-11-29T11:34:25.269 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:21:38.159 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.318 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:21:38.209 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.335 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:21:38.224 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.345 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:21:38.234 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.352 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:21:38.153 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.356 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:21:38.157 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.453 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:21:38.344 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.488 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:21:38.377 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:25.522 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:21:38.323 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:34:26.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:26.080 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-11-29T11:34:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:35.879 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-11-29T11:34:55.181 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:34:55.182 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-11-29T11:35:01.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:35:01.980 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-11-29T11:35:02.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:35:02.580 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-11-29T11:35:03.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:35:03.180 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-11-29T11:35:22.100 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T11:35:22.003+0000 41261700 -1 handle_data failed with exit code -5 2019-11-29T11:35:22.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:35:22.579 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-11-29T11:35:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:35:28.080 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-11-29T11:36:08.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:36:08.680 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-11-29T11:36:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:36:19.680 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-11-29T11:36:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:36:36.780 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-11-29T11:36:43.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:36:43.480 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-11-29T11:38:37.147 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:25:50.037 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.182 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:25:50.071 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.183 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:25:50.073 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.184 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:25:50.073 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.213 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:25:49.994 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.213 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:25:49.997 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.287 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:25:50.177 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.338 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:25:50.123 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:38:37.356 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:25:50.243 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:43:52.180 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:43:52.180 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-11-29T11:43:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:43:58.279 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-11-29T11:44:43.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:44:43.979 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-11-29T11:44:48.781 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:44:48.781 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-11-29T11:44:49.504 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:32:02.394 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.516 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:32:02.406 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.520 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:32:02.410 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.524 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:32:02.302 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.527 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:32:02.417 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.533 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:32:02.311 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.764 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:32:02.542 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.773 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:32:02.663 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:44:49.776 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:32:02.667 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:45:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:18.580 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-11-29T11:45:25.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:25.980 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-11-29T11:45:33.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:33.579 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-11-29T11:45:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:47.180 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-11-29T11:45:48.879 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:48.879 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-11-29T11:45:53.279 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:53.279 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-11-29T11:45:55.879 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:45:55.879 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-11-29T11:46:01.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:01.479 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-11-29T11:46:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:03.379 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-11-29T11:46:10.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:10.680 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-11-29T11:46:14.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:14.480 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-11-29T11:46:22.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:22.679 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-11-29T11:46:26.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:26.179 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-11-29T11:46:33.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:33.679 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-11-29T11:46:37.779 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:37.779 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-11-29T11:46:46.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:46.480 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-11-29T11:46:48.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:48.979 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-11-29T11:46:55.580 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:46:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-11-29T11:47:21.879 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:47:21.879 INFO:teuthology.orchestra.run.smithi069.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-11-29T11:47:28.779 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:47:28.779 INFO:teuthology.orchestra.run.smithi069.stdout:---------------------------------------------------------------------- 2019-11-29T11:47:28.780 INFO:teuthology.orchestra.run.smithi069.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-11-29T11:47:28.780 INFO:teuthology.orchestra.run.smithi069.stdout:---------------------------------------------------------------------- 2019-11-29T11:47:28.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:47:28.979 INFO:teuthology.orchestra.run.smithi069.stdout:BUILD SUCCESSFUL in 19m 28s 2019-11-29T11:47:28.980 INFO:teuthology.orchestra.run.smithi069.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-11-29T11:47:29.300 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:47:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.520+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.544+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.544+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.559 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.557+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.559 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.557+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.561+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.561+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.569+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:29.754 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.751+0000 7fdf146ffec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:47:29.755 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.751+0000 7fdf146ffec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:47:29.755 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.751+0000 7fdf146ffec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:47:29.755 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.751+0000 7fdf146ffec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:47:29.755 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.751+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.765 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.763+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:29.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.763+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.779+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:29.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.779+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.790+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:29.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.790+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.808+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.808+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.816 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.814+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.816 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.814+0000 7fdf146ffec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:47:29.816 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.814+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.820+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.820+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.830 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.828+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:29.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.828+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.846+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:29.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.846+0000 7fdf146ffec0 20 zonegroup default 2019-11-29T11:47:29.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.846+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.857 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.855+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.858 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.855+0000 7fdf146ffec0 10 Cannot find current period zone using local zone 2019-11-29T11:47:29.858 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.855+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.862+0000 7fdf146ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.862+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.871+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:29.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.871+0000 7fdf146ffec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:29.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.887+0000 7fdf146ffec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:29.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.887+0000 7fdf146ffec0 20 zone default found 2019-11-29T11:47:29.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.887+0000 7fdf146ffec0 20 started sync module instance, tier type = 2019-11-29T11:47:29.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:29.887+0000 7fdf146ffec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:47:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=0 2019-11-29T11:47:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=1 2019-11-29T11:47:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=2 2019-11-29T11:47:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=3 2019-11-29T11:47:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=4 2019-11-29T11:47:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.289+0000 7fdf146ffec0 20 add_watcher() i=5 2019-11-29T11:47:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.319+0000 7fdf146ffec0 20 add_watcher() i=6 2019-11-29T11:47:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.323+0000 7fdf146ffec0 20 add_watcher() i=7 2019-11-29T11:47:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.323+0000 7fdf146ffec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:47:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.323+0000 7fded27dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:47:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.326+0000 7fdf0212d700 0 -- >> v1:172.21.15.163:6789/0 conn(0x55772bd050b0 legacy=0x55772bd1a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:47:30.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:30.371+0000 7fdf0212d700 0 -- 172.21.15.69:0/1228812621 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7fde44017570 msgr2=0x7fde44019a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:47:31.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.932+0000 7fde48ff9700 20 reqs_thread_entry: start 2019-11-29T11:47:31.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.932+0000 7fde427fc700 20 reqs_thread_entry: start 2019-11-29T11:47:31.935 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.932+0000 7fdf146ffec0 20 init_complete bucket index max shards: 0 2019-11-29T11:47:31.936 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.932+0000 7fde40ff9700 20 reqs_thread_entry: start 2019-11-29T11:47:31.936 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.932+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:47:31.995 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:31.993+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.023+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.039+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:32.090 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.087+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:47:32.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.189+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:32.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.207+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.621', time=2019-11-29T13:41:44.559035+0000, chain.objs.size()=19 2019-11-29T11:47:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.1 2019-11-29T11:47:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.1_1 2019-11-29T11:47:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.2 2019-11-29T11:47:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.222+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.2_1 2019-11-29T11:47:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.3 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.3_1 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.4 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.4_1 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.5 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.5_1 2019-11-29T11:47:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.6 2019-11-29T11:47:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.223+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.6_1 2019-11-29T11:47:32.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.428+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.7 2019-11-29T11:47:32.486 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.7_1 2019-11-29T11:47:32.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.8 2019-11-29T11:47:32.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.8_1 2019-11-29T11:47:32.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.9 2019-11-29T11:47:32.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.9_1 2019-11-29T11:47:32.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.484+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~JYHX70IRnFT2tUzOI0tAwel6locSVSy.10 2019-11-29T11:47:32.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:32.921 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.918+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:47:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.968+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:32.991+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:33.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:33.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-', time=2019-11-29T13:45:43.184558+0000, chain.objs.size()=35 2019-11-29T11:47:33.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.1 2019-11-29T11:47:33.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.2 2019-11-29T11:47:33.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.3 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.4 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.5 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.6 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.7 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.8 2019-11-29T11:47:33.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.9 2019-11-29T11:47:33.039 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.10 2019-11-29T11:47:33.039 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.11 2019-11-29T11:47:33.039 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.033+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.12 2019-11-29T11:47:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.317+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.13 2019-11-29T11:47:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.328+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.14 2019-11-29T11:47:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.328+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.15 2019-11-29T11:47:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.328+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.agZ5L8d1_C4wWxtIySo4hn4KhqK9Frj.16 2019-11-29T11:47:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.328+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.ecw8EKVIF433p6JEbJzHGoPaJzZIR5m.17 2019-11-29T11:47:33.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.5KJj3D58wh2O4n8QZXSRdX9sVxKecUz.18 2019-11-29T11:47:33.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.HQlX0cAc4jw818_JPiuhu8J255J4cyE.19 2019-11-29T11:47:33.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.608A7rHr6tJsGvHZMXFZemvYDRiGyCW.20 2019-11-29T11:47:33.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.21 2019-11-29T11:47:33.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.22 2019-11-29T11:47:33.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.408+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.23 2019-11-29T11:47:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.482+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.24 2019-11-29T11:47:33.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.600+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.25 2019-11-29T11:47:33.603 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.600+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.26 2019-11-29T11:47:33.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.600+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.27 2019-11-29T11:47:33.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.600+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.28 2019-11-29T11:47:33.848 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.29 2019-11-29T11:47:33.848 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.30 2019-11-29T11:47:33.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.31 2019-11-29T11:47:33.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.32 2019-11-29T11:47:33.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.33 2019-11-29T11:47:33.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.34 2019-11-29T11:47:33.850 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:33.845+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.31__multipart_key1.2~5g6BmiV0ISDoZu4t1EHCalondLiXIu-.35 2019-11-29T11:47:34.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.188+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:34.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.283+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:47:34.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.339+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.359+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:34.392 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:47:34.392 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.596', time=2019-11-29T13:39:21.263872+0000, chain.objs.size()=21 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.1 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.1_1 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.2 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.2_1 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.3 2019-11-29T11:47:34.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.3_1 2019-11-29T11:47:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.4 2019-11-29T11:47:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.4_1 2019-11-29T11:47:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.5 2019-11-29T11:47:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.5_1 2019-11-29T11:47:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.6 2019-11-29T11:47:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.388+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.6_1 2019-11-29T11:47:34.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.691+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.7 2019-11-29T11:47:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.7_1 2019-11-29T11:47:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.8 2019-11-29T11:47:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.8_1 2019-11-29T11:47:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.9 2019-11-29T11:47:34.697 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.9_1 2019-11-29T11:47:34.697 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.693+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.10 2019-11-29T11:47:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.740+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.10_1 2019-11-29T11:47:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.740+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~StApZLw4vzvchQPOucC7hDW1hgYh4E3.11 2019-11-29T11:47:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.840+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.645', time=2019-11-29T13:43:53.549444+0000, chain.objs.size()=16 2019-11-29T11:47:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.840+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.1 2019-11-29T11:47:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:34.840+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.1_1 2019-11-29T11:47:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.1_2 2019-11-29T11:47:35.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.2 2019-11-29T11:47:35.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.2_1 2019-11-29T11:47:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.2_2 2019-11-29T11:47:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.3 2019-11-29T11:47:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.3_1 2019-11-29T11:47:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.077+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.3_2 2019-11-29T11:47:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.127+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.4 2019-11-29T11:47:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.127+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.4_1 2019-11-29T11:47:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.127+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.4_2 2019-11-29T11:47:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.240+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.5 2019-11-29T11:47:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.240+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.5_1 2019-11-29T11:47:35.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.274+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.5_2 2019-11-29T11:47:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.401+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~h1b4v7gwVaxVbCSglbQ1IhL85v-JkUY.6 2019-11-29T11:47:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.767+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:35.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.846+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:47:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.883+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:35.901 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.898+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:35.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.912+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:35.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.912+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.439', time=2019-11-29T13:32:14.693881+0000, chain.objs.size()=9 2019-11-29T11:47:35.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.912+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.1 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.1_1 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.2 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.2_1 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.3 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.3_1 2019-11-29T11:47:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.4 2019-11-29T11:47:35.917 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.4_1 2019-11-29T11:47:35.917 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:35.913+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~XnWraOIgge9gGjJ7J4Vtn4AHO9XP9zE.5 2019-11-29T11:47:36.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.265+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:36.335 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.332+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:47:36.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.365+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:36.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.383+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:36.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:36.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.560', time=2019-11-29T13:36:10.139563+0000, chain.objs.size()=9 2019-11-29T11:47:36.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.1 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.1_1 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.2 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.2_1 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.3 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.3_1 2019-11-29T11:47:36.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.4 2019-11-29T11:47:36.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.4_1 2019-11-29T11:47:36.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.397+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file.mpg.2~jXgPISg7JwKZkfjKYu99khjZW-1acOl.5 2019-11-29T11:47:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.660+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.763+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:47:36.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.803+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.818+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:36.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:36.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.836', time=2019-11-29T13:47:23.201294+0000, chain.objs.size()=21 2019-11-29T11:47:36.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.1 2019-11-29T11:47:36.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.1_1 2019-11-29T11:47:36.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.2 2019-11-29T11:47:36.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.2_1 2019-11-29T11:47:36.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.3 2019-11-29T11:47:36.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.3_1 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.4 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.4_1 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.5 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.5_1 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.6 2019-11-29T11:47:36.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:36.839+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.6_1 2019-11-29T11:47:37.008 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.005+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.7 2019-11-29T11:47:37.071 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.069+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.7_1 2019-11-29T11:47:37.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.081+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.8 2019-11-29T11:47:37.382 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.380+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.8_1 2019-11-29T11:47:37.383 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.380+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.9 2019-11-29T11:47:37.383 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.380+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.9_1 2019-11-29T11:47:37.383 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.380+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.10 2019-11-29T11:47:37.391 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.389+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__shadow_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.10_1 2019-11-29T11:47:37.391 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.389+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.39__multipart_key1.2~gD2NbhuA2-eCntFLiI850KlCsNyK6NN.11 2019-11-29T11:47:37.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.834+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:37.950 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.947+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:47:37.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:37.986+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.004 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.001+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:38.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.009+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.046+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:47:38.313 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.310+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.358 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.354+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.370+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.433+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:47:38.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.534+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.569+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:38.584 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.582+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.627+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:47:38.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.681+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:38.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.699+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.447', time=2019-11-29T13:32:45.851158+0000, chain.objs.size()=9 2019-11-29T11:47:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.1 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.1_1 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.2 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.2_1 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.3 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.3_1 2019-11-29T11:47:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.4 2019-11-29T11:47:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.4_1 2019-11-29T11:47:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:38.714+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~Y_YilTCqq3VxbCps9rVPcMiM4Pl1GKc.5 2019-11-29T11:47:39.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.124+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:39.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.218+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:47:39.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.265+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.283+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:39.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:39.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.675', time=2019-11-29T13:45:20.601306+0000, chain.objs.size()=9 2019-11-29T11:47:39.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__multipart_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.1 2019-11-29T11:47:39.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__shadow_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.1_1 2019-11-29T11:47:39.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__multipart_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.2 2019-11-29T11:47:39.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__shadow_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.2_1 2019-11-29T11:47:39.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__multipart_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.3 2019-11-29T11:47:39.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__shadow_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.3_1 2019-11-29T11:47:39.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__multipart_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.4 2019-11-29T11:47:39.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__shadow_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.4_1 2019-11-29T11:47:39.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.298+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.30__multipart_key1.2~UvXBjH3FPBFzouhi1U0kR14rL_Edl0i.5 2019-11-29T11:47:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.564+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:39.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.692+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:47:39.736 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.733+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:39.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.749+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:39.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:39.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT', time=2019-11-29T13:28:58.942397+0000, chain.objs.size()=9 2019-11-29T11:47:39.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__multipart_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.1 2019-11-29T11:47:39.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__shadow_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.1_1 2019-11-29T11:47:39.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__multipart_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.2 2019-11-29T11:47:39.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__shadow_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.2_1 2019-11-29T11:47:39.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__multipart_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.3 2019-11-29T11:47:39.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__shadow_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.3_1 2019-11-29T11:47:39.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__multipart_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.4 2019-11-29T11:47:39.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__shadow_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.4_1 2019-11-29T11:47:39.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:39.763+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.13__multipart_key1.2~LYOQN_dwcDn1Ypt1D8nI3gY_z2e8uvT.5 2019-11-29T11:47:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.073+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:40.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.287+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:47:40.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.339+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.364 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.361+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:40.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.372+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.418 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.415+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:47:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.480+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.508+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:40.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:40.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.431', time=2019-11-29T13:34:00.828196+0000, chain.objs.size()=5 2019-11-29T11:47:40.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.19__shadow_.hCt0pjSyx-umtN8IdlvBKoPLWctMQiJ_1 2019-11-29T11:47:40.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.19__shadow_.hCt0pjSyx-umtN8IdlvBKoPLWctMQiJ_2 2019-11-29T11:47:40.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.19__shadow_.hCt0pjSyx-umtN8IdlvBKoPLWctMQiJ_3 2019-11-29T11:47:40.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.19__shadow_.hCt0pjSyx-umtN8IdlvBKoPLWctMQiJ_4 2019-11-29T11:47:40.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.525+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.19__shadow_.hCt0pjSyx-umtN8IdlvBKoPLWctMQiJ_5 2019-11-29T11:47:40.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.735+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:40.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.805+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:47:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.843+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.863+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:40.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.871+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.907 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.906+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:47:40.948 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.946+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:40.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.961+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.978+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.978+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.455', time=2019-11-29T13:33:10.848080+0000, chain.objs.size()=9 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.978+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.1 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.978+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.1_1 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.978+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.2 2019-11-29T11:47:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.2_1 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.3 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.3_1 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.4 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.4_1 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~OIH21CH8V3sja-LL-BxMUcJCsaBJlQ3.5 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.561', time=2019-11-29T13:36:12.500420+0000, chain.objs.size()=9 2019-11-29T11:47:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.1 2019-11-29T11:47:40.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.1_1 2019-11-29T11:47:40.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:40.979+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.2 2019-11-29T11:47:41.098 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.096+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.2_1 2019-11-29T11:47:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.235+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.3 2019-11-29T11:47:41.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.274+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.3_1 2019-11-29T11:47:41.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.274+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.4 2019-11-29T11:47:41.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.373+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__shadow_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.4_1 2019-11-29T11:47:41.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.373+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.26__multipart_file2.mpg.2~5bgmyAn3ghco6EXpx_BA7QR0vSlKAhX.5 2019-11-29T11:47:41.701 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.699+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:41.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.888+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:47:41.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.952+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.971+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.467', time=2019-11-29T13:33:57.310270+0000, chain.objs.size()=7 2019-11-29T11:47:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.1 2019-11-29T11:47:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.1_1 2019-11-29T11:47:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.1_2 2019-11-29T11:47:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.2 2019-11-29T11:47:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.2_1 2019-11-29T11:47:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.2_2 2019-11-29T11:47:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:41.986+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~1Kh6joiQ9v8RqXgepnN_mHqc8MfHxs2.3 2019-11-29T11:47:42.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.247+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:42.332 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.330+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:47:42.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.368+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:42.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.409+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:42.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:47:42.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.416', time=2019-11-29T13:30:22.605246+0000, chain.objs.size()=9 2019-11-29T11:47:42.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.1 2019-11-29T11:47:42.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.1_1 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.2 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.2_1 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.3 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.3_1 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.4 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.4_1 2019-11-29T11:47:42.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.5 2019-11-29T11:47:42.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.418', time=2019-11-29T13:30:31.048559+0000, chain.objs.size()=9 2019-11-29T11:47:42.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.1 2019-11-29T11:47:42.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.1_1 2019-11-29T11:47:42.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.426+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.2 2019-11-29T11:47:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.539+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.2_1 2019-11-29T11:47:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.539+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.3 2019-11-29T11:47:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.539+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.3_1 2019-11-29T11:47:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.788+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.4 2019-11-29T11:47:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.788+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__shadow_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.4_1 2019-11-29T11:47:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:42.788+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.17__multipart_key1.2~CXjE2TMjNljEBn3Y-wABfy3Fk6jHAD6.5 2019-11-29T11:47:43.031 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.029+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:43.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.141+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:47:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.182+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:43.202 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.201+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.209+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.236+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:47:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.271+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:43.287 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.286+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:43.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.294+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:43.322 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.321+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:47:43.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.404+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:43.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.440+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.609', time=2019-11-29T13:40:39.685006+0000, chain.objs.size()=21 2019-11-29T11:47:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.1 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.1_1 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.2 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.2_1 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.3 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.3_1 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.4 2019-11-29T11:47:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.4_1 2019-11-29T11:47:43.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.5 2019-11-29T11:47:43.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.5_1 2019-11-29T11:47:43.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.6 2019-11-29T11:47:43.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.476+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.6_1 2019-11-29T11:47:43.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.668+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.7 2019-11-29T11:47:43.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:43.831+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.7_1 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.8 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.8_1 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.9 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.9_1 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.10 2019-11-29T11:47:44.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.10_1 2019-11-29T11:47:44.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.038+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~yoGR7hT2L142ber8GG7GgS9Z7YJQv58.11 2019-11-29T11:47:44.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.359+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:44.550 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.549+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:47:44.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.591+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:44.608 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.606+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:44.623 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.461', time=2019-11-29T13:33:34.334268+0000, chain.objs.size()=7 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.1 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.1_1 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.1_2 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.2 2019-11-29T11:47:44.624 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.2_1 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.2_2 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~_jWxJWwOVr0xHLY4Xa9ssom8IEKfeN7.3 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.510', time=2019-11-29T13:34:57.325653+0000, chain.objs.size()=9 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__multipart_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.1 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__shadow_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.1_1 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__multipart_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.2 2019-11-29T11:47:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__shadow_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.2_1 2019-11-29T11:47:44.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.621+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__multipart_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.3 2019-11-29T11:47:44.995 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.993+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__shadow_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.3_1 2019-11-29T11:47:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.993+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__multipart_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.4 2019-11-29T11:47:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.993+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__shadow_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.4_1 2019-11-29T11:47:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:44.993+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.24__multipart_key1.2~kw3mItpLYiacvKD_CTQQIWMyrLRaWjq.5 2019-11-29T11:47:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.257+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:45.351 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.346+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:47:45.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.395+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:45.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.412+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:45.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-29T11:47:45.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.473', time=2019-11-29T13:34:06.921558+0000, chain.objs.size()=7 2019-11-29T11:47:45.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.1 2019-11-29T11:47:45.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.1_1 2019-11-29T11:47:45.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.1_2 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.2 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.2_1 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__shadow_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.2_2 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.20__multipart_key1.2~FgudS7Tlm2CTF7uDRYj-jIdw1o3byYk.3 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N', time=2019-11-29T13:36:40.381002+0000, chain.objs.size()=5 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.27__multipart_key1.2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N.7 2019-11-29T11:47:45.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.27__shadow_key1.2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N.7_1 2019-11-29T11:47:45.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.27__multipart_key1.2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N.10 2019-11-29T11:47:45.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.27__shadow_key1.2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N.10_1 2019-11-29T11:47:45.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.433+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.27__multipart_key1.2~rjfAidfpPOwNPHicYuQM7fYSDXacx4N.13 2019-11-29T11:47:45.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.645+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.629', time=2019-11-29T13:42:49.387642+0000, chain.objs.size()=16 2019-11-29T11:47:45.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.645+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.1 2019-11-29T11:47:45.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.645+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.1_1 2019-11-29T11:47:45.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.647+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.1_2 2019-11-29T11:47:45.681 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.680+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.2 2019-11-29T11:47:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.2_1 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.2_2 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.3 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.3_1 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.3_2 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.4 2019-11-29T11:47:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.859+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.4_1 2019-11-29T11:47:45.922 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:45.921+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.4_2 2019-11-29T11:47:46.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.001+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.5 2019-11-29T11:47:46.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.001+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.5_1 2019-11-29T11:47:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.097+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.5_2 2019-11-29T11:47:46.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.256+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~evCN5rEqWlfX_mT_moMK_CJqVPXlKR4.6 2019-11-29T11:47:46.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.518+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:46.616 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.615+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:47:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.670+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:46.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.689+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:46.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.697+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:46.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.726+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:47:46.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.774+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:46.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.801+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:46.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:46.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.387', time=2019-11-29T13:29:43.001373+0000, chain.objs.size()=9 2019-11-29T11:47:46.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__multipart_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.1 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__shadow_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.1_1 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__multipart_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.2 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__shadow_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.2_1 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__multipart_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.3 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__shadow_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.3_1 2019-11-29T11:47:46.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__multipart_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.4 2019-11-29T11:47:46.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__shadow_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.4_1 2019-11-29T11:47:46.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:46.820+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.15__multipart_key1.2~kgRGYzfC9QtzEF8YAQD_qaZuqx55ljj.5 2019-11-29T11:47:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.218+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:47.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.304+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:47:47.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.358+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.379 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.378+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:47.388 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.387+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.435+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:47:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.475+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.494 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.493+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:47.506 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.505+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.547 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.545+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:47:47.606 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.604+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.633+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.649+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.706 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.704+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:47:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.793+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:47.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.817+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.529', time=2019-11-29T13:35:24.267673+0000, chain.objs.size()=9 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__multipart_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.1 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__shadow_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.1_1 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__multipart_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.2 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__shadow_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.2_1 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__multipart_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.3 2019-11-29T11:47:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__shadow_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.3_1 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__multipart_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.4 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__shadow_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.4_1 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.25__multipart_key1.2~RNoCR91AQauxQsgJU8OYKQMAUBGCJvC.5 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.637', time=2019-11-29T13:43:50.306011+0000, chain.objs.size()=16 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.1 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.1_1 2019-11-29T11:47:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:47.838+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.1_2 2019-11-29T11:47:48.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.128+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.2 2019-11-29T11:47:48.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.128+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.2_1 2019-11-29T11:47:48.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.128+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.2_2 2019-11-29T11:47:48.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.3 2019-11-29T11:47:48.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.3_1 2019-11-29T11:47:48.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.3_2 2019-11-29T11:47:48.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.4 2019-11-29T11:47:48.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.4_1 2019-11-29T11:47:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.4_2 2019-11-29T11:47:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.5 2019-11-29T11:47:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.149+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.5_1 2019-11-29T11:47:48.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.552+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__shadow_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.5_2 2019-11-29T11:47:48.554 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.552+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.28__multipart_key1.2~hcxZVDYQ6dyaESLUzVVBQH7V2cK2ryw.6 2019-11-29T11:47:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.774+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:48.857 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.855+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:47:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.888+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:48.904 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.902+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:48.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.910+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:48.941 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.940+0000 7fdf146ffec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:47:48.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:48.980+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:49.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.008+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:49.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.660', time=2019-11-29T13:44:45.194344+0000, chain.objs.size()=9 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__multipart_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.1 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__shadow_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.1_1 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__multipart_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.2 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__shadow_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.2_1 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__multipart_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.3 2019-11-29T11:47:49.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__shadow_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.3_1 2019-11-29T11:47:49.031 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__multipart_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.4 2019-11-29T11:47:49.031 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__shadow_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.4_1 2019-11-29T11:47:49.031 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.027+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.29__multipart_key1.2~_KMsGZzIK_tCvmu9TwwHz5s0Av-HDYd.5 2019-11-29T11:47:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.394+0000 7fdf146ffec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T11:47:49.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.479+0000 7fdf146ffec0 20 remove_watcher() i=0 2019-11-29T11:47:49.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.479+0000 7fdf146ffec0 2 removed watcher, disabling cache 2019-11-29T11:47:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.510+0000 7fdf146ffec0 20 remove_watcher() i=1 2019-11-29T11:47:49.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.543+0000 7fdf146ffec0 20 remove_watcher() i=2 2019-11-29T11:47:49.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.571+0000 7fdf146ffec0 20 remove_watcher() i=3 2019-11-29T11:47:49.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.596+0000 7fdf146ffec0 20 remove_watcher() i=4 2019-11-29T11:47:49.623 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.621+0000 7fdf146ffec0 20 remove_watcher() i=5 2019-11-29T11:47:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.656+0000 7fdf146ffec0 20 remove_watcher() i=6 2019-11-29T11:47:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.684+0000 7fdf146ffec0 20 remove_watcher() i=7 2019-11-29T11:47:49.693 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:47:49.800 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.798+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:49.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.822+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:49.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.822+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:49.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.831+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:49.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.831+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:49.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.837+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:49.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.837+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:49.844+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.063+0000 7f10c7ad6ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:47:50.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.063+0000 7f10c7ad6ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:47:50.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.063+0000 7f10c7ad6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:47:50.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.063+0000 7f10c7ad6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:47:50.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.063+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.074 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.073+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:50.074 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.073+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.092+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.092+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.111+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.111+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.129+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.129+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.148+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:50.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.148+0000 7f10c7ad6ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:47:50.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.148+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.161 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.160+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:50.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.160+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.176+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.176+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.193+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.193+0000 7f10c7ad6ec0 20 zonegroup default 2019-11-29T11:47:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.193+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.200+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:50.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.200+0000 7f10c7ad6ec0 10 Cannot find current period zone using local zone 2019-11-29T11:47:50.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.200+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.206+0000 7f10c7ad6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:50.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.206+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.219 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.217+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:50.219 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.217+0000 7f10c7ad6ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:50.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.236+0000 7f10c7ad6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:50.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.236+0000 7f10c7ad6ec0 20 zone default found 2019-11-29T11:47:50.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.236+0000 7f10c7ad6ec0 20 started sync module instance, tier type = 2019-11-29T11:47:50.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.236+0000 7f10c7ad6ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:47:50.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=0 2019-11-29T11:47:50.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=1 2019-11-29T11:47:50.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=2 2019-11-29T11:47:50.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=3 2019-11-29T11:47:50.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=4 2019-11-29T11:47:50.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=5 2019-11-29T11:47:50.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.694+0000 7f10c7ad6ec0 20 add_watcher() i=6 2019-11-29T11:47:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.723+0000 7f10c7ad6ec0 20 add_watcher() i=7 2019-11-29T11:47:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.723+0000 7f10c7ad6ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:47:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.724+0000 7f10857e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:47:50.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.728+0000 7f10b4d03700 0 -- >> v2:172.21.15.69:3300/0 conn(0x55caa8b7ad70 msgr2=0x55caa8b7b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-29T11:47:50.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:50.772+0000 7f10b4502700 0 -- 172.21.15.69:0/105886788 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x55caa8bc8b30 msgr2=0x55caa8bcaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T11:47:52.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.373+0000 7f0ff77fe700 20 reqs_thread_entry: start 2019-11-29T11:47:52.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.373+0000 7f0ff57fa700 20 reqs_thread_entry: start 2019-11-29T11:47:52.376 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.373+0000 7f10c7ad6ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:47:52.376 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.374+0000 7f0fe37fe700 20 reqs_thread_entry: start 2019-11-29T11:47:52.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.374+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:47:52.421 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.419+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.438 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.436+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:52.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.445+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.473 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.472+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:47:52.562 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.561+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.600 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.599+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:52.623 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.621+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.682 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.677+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:47:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.744+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.763+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.777+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.820 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.818+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:47:52.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.892+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.936+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.948+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:52.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:52.988+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:47:53.026 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.025+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.042 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.040+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.048+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.084+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:47:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.120+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.135+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.143+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.176 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.175+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:47:53.222 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.220+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.239+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.251 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.249+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.278+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:47:53.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.377+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.404 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.403+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.414+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.460+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:47:53.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.521+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.551+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.559+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.605 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.604+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:47:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.648+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.667+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.675+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.706+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:47:53.753 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.752+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.771+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.785 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.783+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.813+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:47:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.849+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.867 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.865+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.876+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:53.913 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.911+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:47:53.997 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:53.996+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.029+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.037+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.088 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.086+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:47:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.132+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.149+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.159+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.209+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:47:54.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.268+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.294+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.307+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.353 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.351+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:47:54.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.402+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.423 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.422+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.432+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.459+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:47:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.507+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.526+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.537 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.535+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.572+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:47:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.625+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.652 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.650+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.664 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.661+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.699 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.698+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:47:54.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.744+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.763 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.761+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.771+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.813+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:47:54.874 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.873+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.893+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:54.906 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.904+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:54.944 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:54.942+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:47:55.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.026+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.060+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.075 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.073+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.124+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:47:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.189+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.214+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.224 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.223+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.258+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:47:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.299+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.317 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.316+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.326 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.325+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.354+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:47:55.421 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.420+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.444+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.460 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.458+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.511+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:47:55.559 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.558+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.578 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.576+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.586+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.624+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:47:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.698+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.719 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.717+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.727+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.769+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:47:55.830 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.828+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.862+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:55.874 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.873+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.920+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:47:55.976 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.974+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:55.993+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.001+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.039+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:47:56.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.091+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.107+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.116+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.159+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:47:56.244 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.242+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.271 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.269+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:56.282 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.280+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.338 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.337+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:47:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.383+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.401+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:56.412 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.411+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.460+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:47:56.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.560+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.595 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.593+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:56.612 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.610+0000 7f10c7ad6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:56.703 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.702+0000 7f10c7ad6ec0 20 remove_watcher() i=0 2019-11-29T11:47:56.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.702+0000 7f10c7ad6ec0 2 removed watcher, disabling cache 2019-11-29T11:47:56.762 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.760+0000 7f10c7ad6ec0 20 remove_watcher() i=1 2019-11-29T11:47:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.814+0000 7f10c7ad6ec0 20 remove_watcher() i=2 2019-11-29T11:47:56.878 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.876+0000 7f10c7ad6ec0 20 remove_watcher() i=3 2019-11-29T11:47:56.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.924+0000 7f10c7ad6ec0 20 remove_watcher() i=4 2019-11-29T11:47:56.977 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:56.975+0000 7f10c7ad6ec0 20 remove_watcher() i=5 2019-11-29T11:47:57.038 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.036+0000 7f10c7ad6ec0 20 remove_watcher() i=6 2019-11-29T11:47:57.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.090+0000 7f10c7ad6ec0 20 remove_watcher() i=7 2019-11-29T11:47:57.105 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:47:57.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.267+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.294+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.294+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.304+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.304+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.310 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.309+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.311 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.309+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.317+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:57.526 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.524+0000 7efdba0e4ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:47:57.526 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.524+0000 7efdba0e4ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:47:57.526 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.524+0000 7efdba0e4ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:47:57.526 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.524+0000 7efdba0e4ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:47:57.526 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.524+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.533+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:57.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.533+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.548+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:57.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.548+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.560 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.558+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:57.560 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.558+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.570+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:57.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.570+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.584+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.584+0000 7efdba0e4ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:47:57.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.584+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.589+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.589+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.596+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:57.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.596+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.612+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:47:57.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.612+0000 7efdba0e4ec0 20 zonegroup default 2019-11-29T11:47:57.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.612+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.617+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.617+0000 7efdba0e4ec0 10 Cannot find current period zone using local zone 2019-11-29T11:47:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.617+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.623+0000 7efdba0e4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:47:57.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.623+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.635 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.634+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:47:57.635 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.634+0000 7efdba0e4ec0 20 rados->read ofs=0 len=0 2019-11-29T11:47:57.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.652+0000 7efdba0e4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:47:57.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.653+0000 7efdba0e4ec0 20 zone default found 2019-11-29T11:47:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.653+0000 7efdba0e4ec0 20 started sync module instance, tier type = 2019-11-29T11:47:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:57.653+0000 7efdba0e4ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=0 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=1 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=2 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=3 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=4 2019-11-29T11:47:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=5 2019-11-29T11:47:58.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.019+0000 7efdba0e4ec0 20 add_watcher() i=6 2019-11-29T11:47:58.055 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.054+0000 7efdba0e4ec0 20 add_watcher() i=7 2019-11-29T11:47:58.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.054+0000 7efdba0e4ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:47:58.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.054+0000 7efd77fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:47:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.064+0000 7efda7b12700 0 -- 172.21.15.69:0/3765039153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a19c0dd70 msgr2=0x557a19c23290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:47:58.109 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:58.107+0000 7efda7311700 0 -- 172.21.15.69:0/2174679459 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7efce80175a0 msgr2=0x7efce8019a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:47:59.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.816+0000 7efce67fc700 20 reqs_thread_entry: start 2019-11-29T11:47:59.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.816+0000 7efcdffff700 20 reqs_thread_entry: start 2019-11-29T11:47:59.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.816+0000 7efdba0e4ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:47:59.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.817+0000 7efcde7fc700 20 reqs_thread_entry: start 2019-11-29T11:47:59.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.817+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:47:59.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.861+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:59.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.884+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:47:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.896+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:47:59.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.935+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:48:00.001 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:47:59.999+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.026+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.036 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.034+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.078+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:48:00.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.136+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.157+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.170+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.217+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:48:00.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.281+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.307 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.305+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.321 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.319+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.358+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:48:00.434 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.432+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.460+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.475 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.473+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.512+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:48:00.577 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.575+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.603+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.617+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.657+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:48:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.708+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.729+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.742 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.740+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.776+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:48:00.869 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.867+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.908 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.906+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.922+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:00.997 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:00.996+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:48:01.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.081+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.112+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.134+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.187+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:48:01.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.252+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.274 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.272+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.282 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.280+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.330 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.328+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:48:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.412+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.440+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.450+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.506+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:48:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.627+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.646+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.656 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.655+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.686 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.685+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:48:01.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.746+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.774 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.772+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.785 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.784+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.832+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:48:01.919 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.917+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:01.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.953+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:01.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:01.980+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.037 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.036+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:48:02.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.095+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.115+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.128+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.181+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:48:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.259+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.323 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.322+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.345+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.405 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.404+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:48:02.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.505+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.551+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.563+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.594 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.592+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:48:02.666 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.664+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.703 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.702+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.720+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.769 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.768+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:48:02.812 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.811+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.828 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.826+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.835+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.875+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:48:02.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.921+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.937+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:02.952+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.023+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:48:03.074 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.072+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.094 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.092+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.102+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.137+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:48:03.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.179+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.198 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.197+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.207 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.205+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.240 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.239+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:48:03.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.275+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.293+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.304+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.349 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.347+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.424+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.453+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.467+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.520+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:48:03.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.595+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.624+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.638+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.695+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:48:03.764 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.762+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.783 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.782+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.792+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.831+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:48:03.875 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.874+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.892 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.891+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.901 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.899+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.930+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:48:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.969+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:03.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.984+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:03.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:03.995+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.043+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:48:04.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.099+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.128+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:04.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.137+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.172+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:48:04.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.224+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.241+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.250+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.298+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:48:04.361 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.359+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.380 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.379+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:04.389 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.387+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.443 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.442+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:48:04.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.516+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.544+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.556+0000 7efdba0e4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:04.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.675+0000 7efdba0e4ec0 20 remove_watcher() i=0 2019-11-29T11:48:04.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.675+0000 7efdba0e4ec0 2 removed watcher, disabling cache 2019-11-29T11:48:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.723+0000 7efdba0e4ec0 20 remove_watcher() i=1 2019-11-29T11:48:04.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.758+0000 7efdba0e4ec0 20 remove_watcher() i=2 2019-11-29T11:48:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.822+0000 7efdba0e4ec0 20 remove_watcher() i=3 2019-11-29T11:48:04.876 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.875+0000 7efdba0e4ec0 20 remove_watcher() i=4 2019-11-29T11:48:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.918+0000 7efdba0e4ec0 20 remove_watcher() i=5 2019-11-29T11:48:04.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:04.974+0000 7efdba0e4ec0 20 remove_watcher() i=6 2019-11-29T11:48:05.033 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.031+0000 7efdba0e4ec0 20 remove_watcher() i=7 2019-11-29T11:48:05.038 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:48:05.351 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.349+0000 7f2c0940a700 0 --2- 172.21.15.69:0/3959353328 >> v2:172.21.15.69:3300/0 conn(0x5616a89f3120 0x5616a8ad9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-29T11:48:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.373+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.412+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.412+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.426+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.426+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.434 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.433+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.433+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.448+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:48:05.680 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.678+0000 7f2c1c1ddec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:48:05.680 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.678+0000 7f2c1c1ddec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:48:05.680 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.678+0000 7f2c1c1ddec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:48:05.680 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.678+0000 7f2c1c1ddec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:48:05.681 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.678+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.692+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:48:05.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.692+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.712+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:48:05.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.712+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.726+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:48:05.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.726+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.743+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:48:05.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.743+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.750+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.750+0000 7f2c1c1ddec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:48:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.750+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.758+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.760 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.758+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.772+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:48:05.775 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.772+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.790+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:48:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.790+0000 7f2c1c1ddec0 20 zonegroup default 2019-11-29T11:48:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.790+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.798+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.798+0000 7f2c1c1ddec0 10 Cannot find current period zone using local zone 2019-11-29T11:48:05.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.798+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.805+0000 7f2c1c1ddec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:48:05.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.805+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.820+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:48:05.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.820+0000 7f2c1c1ddec0 20 rados->read ofs=0 len=0 2019-11-29T11:48:05.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.836+0000 7f2c1c1ddec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:48:05.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.836+0000 7f2c1c1ddec0 20 zone default found 2019-11-29T11:48:05.838 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.836+0000 7f2c1c1ddec0 20 started sync module instance, tier type = 2019-11-29T11:48:05.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:05.836+0000 7f2c1c1ddec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:48:06.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=0 2019-11-29T11:48:06.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=1 2019-11-29T11:48:06.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=2 2019-11-29T11:48:06.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=3 2019-11-29T11:48:06.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=4 2019-11-29T11:48:06.216 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.213+0000 7f2c1c1ddec0 20 add_watcher() i=5 2019-11-29T11:48:06.220 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.218+0000 7f2c1c1ddec0 20 add_watcher() i=6 2019-11-29T11:48:06.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.254+0000 7f2c1c1ddec0 20 add_watcher() i=7 2019-11-29T11:48:06.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.254+0000 7f2c1c1ddec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:48:06.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.254+0000 7f2bd9fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:48:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.257+0000 7f2c09c0b700 0 -- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5616a8b10150 msgr2=0x5616a8b418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:48:06.297 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:06.296+0000 7f2c09c0b700 0 -- 172.21.15.69:0/3598636218 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7f2b4c017580 msgr2=0x7f2b4c019a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:48:07.897 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.895+0000 7f2b43fff700 20 reqs_thread_entry: start 2019-11-29T11:48:07.898 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.895+0000 7f2b41ffb700 20 reqs_thread_entry: start 2019-11-29T11:48:07.898 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.895+0000 7f2c1c1ddec0 20 init_complete bucket index max shards: 0 2019-11-29T11:48:07.899 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.896+0000 7f2b33fff700 20 reqs_thread_entry: start 2019-11-29T11:48:07.899 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.897+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:48:07.943 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.941+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:07.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.969+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:07.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:07.982+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.014 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.012+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:48:08.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.086+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.124+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.139+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.200 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.197+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:48:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.240+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.258+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.267+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.307 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.305+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:48:08.352 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.349+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.367+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.375+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.407+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:48:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.488+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.522+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.539+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.604 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.601+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:48:08.687 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.685+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.712+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.729+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.773+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:48:08.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.840+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.861+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:08.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.870+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.929+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:48:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:08.989+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.017+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.033 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.031+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.080+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:48:09.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.167+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.228+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.240+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.280+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:48:09.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.331+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.373+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.388 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.385+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.425+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:48:09.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.511+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.525+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.533+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.568 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.566+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:48:09.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.622+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.641+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.653 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.651+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.697 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.695+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:48:09.754 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.752+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.773+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.785 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.782+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.829+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:48:09.879 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.877+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.892+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:09.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.900+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:09.938 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.935+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:48:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:09.979+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.000+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.008+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.038+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:48:10.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.087+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.106+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.116 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.114+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.142+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:48:10.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.186+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.203+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.212+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.243+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:48:10.330 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.327+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.372 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.370+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.389 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.386+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.439 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.436+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:48:10.498 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.495+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.534 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.531+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.541+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.594 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.591+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:48:10.644 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.641+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.659+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.676+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.709+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:48:10.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.767+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.793 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.791+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.807+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.842+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:48:10.894 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.891+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.913 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.911+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:10.921 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.919+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:10.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.949+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:48:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:10.997+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.015+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.023+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.058+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:48:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.115+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.131+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.141 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.139+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.180+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:48:11.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.222+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.243+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.252+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.311 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.308+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:48:11.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.344+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.359+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.366+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.401+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:48:11.512 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.509+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.532+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.541+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.573+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:48:11.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.622+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.639+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.653 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.650+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.705+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:48:11.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.775+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.806 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.804+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.817 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.815+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.870 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.868+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:48:11.908 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.905+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.921+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:11.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.928+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:11.957+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:48:12.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.041+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:12.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.081+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:12.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.098+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:12.160 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.157+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:48:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.208+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:12.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.229+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:48:12.251 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.248+0000 7f2c1c1ddec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:48:12.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.328+0000 7f2c1c1ddec0 20 remove_watcher() i=0 2019-11-29T11:48:12.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.328+0000 7f2c1c1ddec0 2 removed watcher, disabling cache 2019-11-29T11:48:12.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.374+0000 7f2c1c1ddec0 20 remove_watcher() i=1 2019-11-29T11:48:12.411 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.408+0000 7f2c1c1ddec0 20 remove_watcher() i=2 2019-11-29T11:48:12.464 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.462+0000 7f2c1c1ddec0 20 remove_watcher() i=3 2019-11-29T11:48:12.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.518+0000 7f2c1c1ddec0 20 remove_watcher() i=4 2019-11-29T11:48:12.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.588+0000 7f2c1c1ddec0 20 remove_watcher() i=5 2019-11-29T11:48:12.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.647+0000 7f2c1c1ddec0 20 remove_watcher() i=6 2019-11-29T11:48:12.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:48:12.710+0000 7f2c1c1ddec0 20 remove_watcher() i=7 2019-11-29T11:48:12.721 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-11-29T11:48:13.637 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :clean 2019-11-29T11:48:14.234 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileJava 2019-11-29T11:48:14.234 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processResources 2019-11-29T11:48:14.234 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :classes 2019-11-29T11:48:14.534 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:14.535 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileTestJava 2019-11-29T11:48:14.535 INFO:teuthology.orchestra.run.smithi069.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-29T11:48:14.535 INFO:teuthology.orchestra.run.smithi069.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-29T11:48:14.535 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:14.536 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processTestResources NO-SOURCE 2019-11-29T11:48:14.536 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :testClasses 2019-11-29T11:48:17.134 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :test 2019-11-29T11:48:17.135 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-11-29T11:48:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:19.234 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-11-29T11:48:24.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-11-29T11:48:26.136 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:26.136 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-11-29T11:48:32.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:32.934 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-11-29T11:48:35.434 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:35.434 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-11-29T11:48:42.634 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:42.634 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-11-29T11:48:48.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:48.934 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-11-29T11:48:50.334 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:50.334 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-11-29T11:48:55.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:48:55.934 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-11-29T11:49:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:03.934 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-11-29T11:49:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:04.134 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-11-29T11:49:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:05.034 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-11-29T11:49:11.334 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:11.334 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-11-29T11:49:23.634 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:23.634 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-11-29T11:49:27.534 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:27.534 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-11-29T11:49:32.534 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:32.534 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-11-29T11:49:33.334 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:33.334 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-11-29T11:49:34.135 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:34.135 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------- 2019-11-29T11:49:34.135 INFO:teuthology.orchestra.run.smithi069.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-11-29T11:49:34.135 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------- 2019-11-29T11:49:34.151 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:49:34.151 INFO:teuthology.orchestra.run.smithi069.stdout:BUILD SUCCESSFUL in 1m 21s 2019-11-29T11:49:34.152 INFO:teuthology.orchestra.run.smithi069.stdout:5 actionable tasks: 5 executed 2019-11-29T11:49:34.158 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:49:34.315 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.313+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.332+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.332+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.343 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.341+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.343 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.341+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.346+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.346+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.356+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:34.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.578+0000 7f91d243bec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:49:34.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.578+0000 7f91d243bec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:49:34.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.578+0000 7f91d243bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:49:34.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.578+0000 7f91d243bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:49:34.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.578+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.587+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.587+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.605+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.605+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.616+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.616+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.638 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.636+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:34.638 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.636+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.645+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.645+0000 7f91d243bec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:49:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.645+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.652+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.654 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.652+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.666+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.666+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.688 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.686+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:34.689 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.686+0000 7f91d243bec0 20 zonegroup default 2019-11-29T11:49:34.689 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.686+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.694+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.694+0000 7f91d243bec0 10 Cannot find current period zone using local zone 2019-11-29T11:49:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.694+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.710+0000 7f91d243bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.710+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.722+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:34.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.722+0000 7f91d243bec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.738+0000 7f91d243bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.738+0000 7f91d243bec0 20 zone default found 2019-11-29T11:49:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.738+0000 7f91d243bec0 20 started sync module instance, tier type = 2019-11-29T11:49:34.741 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:34.738+0000 7f91d243bec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:49:35.207 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=0 2019-11-29T11:49:35.207 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=1 2019-11-29T11:49:35.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=2 2019-11-29T11:49:35.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=3 2019-11-29T11:49:35.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=4 2019-11-29T11:49:35.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=5 2019-11-29T11:49:35.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.205+0000 7f91d243bec0 20 add_watcher() i=6 2019-11-29T11:49:35.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.271+0000 7f91d243bec0 20 add_watcher() i=7 2019-11-29T11:49:35.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.271+0000 7f91d243bec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:49:35.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.271+0000 7f918efed700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:49:35.278 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.276+0000 7f91bee67700 0 -- 172.21.15.69:0/68940600 >> v2:172.21.15.69:3300/0 conn(0x5585a09b40b0 msgr2=0x5585a09b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:49:35.325 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:35.323+0000 7f91bf668700 0 -- 172.21.15.69:0/3326014243 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9100003ef0 msgr2=0x7f9100002190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:49:37.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.078+0000 7f90fd7fa700 20 reqs_thread_entry: start 2019-11-29T11:49:37.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.079+0000 7f90f6ffd700 20 reqs_thread_entry: start 2019-11-29T11:49:37.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.079+0000 7f91d243bec0 20 init_complete bucket index max shards: 0 2019-11-29T11:49:37.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.079+0000 7f90f57fa700 20 reqs_thread_entry: start 2019-11-29T11:49:37.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.080+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:49:37.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.132+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.148+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.156+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.213 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.211+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:49:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.268+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.289+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.299+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.344+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:49:37.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.400+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.427 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.425+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.438 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.437+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.471+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:49:37.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.525+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.540+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.551 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.550+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.587+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:49:37.665 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.663+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.703 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.702+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.716 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.715+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.765 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.763+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:49:37.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.821+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.841+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:37.855 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.853+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:37.900 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:37.898+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:49:38.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.000+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.039+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.058 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.057+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.122 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.121+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:49:38.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.188+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.212 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.211+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.226 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.224+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.264+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:49:38.335 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.334+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.358+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.369+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.414+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:49:38.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.475+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.506+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.521+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.568 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.566+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:49:38.636 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.628+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.652 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.649+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.660+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.813 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.810+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:49:38.878 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.876+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.902 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.900+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:38.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.915+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:38.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.952+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:49:39.001 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:38.999+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.021 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.019+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.033 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.031+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.073+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:49:39.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.116+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.133 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.131+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.141+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.168+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:49:39.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.253+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.291 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.288+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.304+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.348+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:49:39.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.399+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.424 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.421+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.433+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.473 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.471+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:49:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.526+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.549 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.547+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.555+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.604+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:49:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.652+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.674+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.687 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.684+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.720+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:49:39.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.766+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.784+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.797 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.795+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.828 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.825+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:49:39.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.884+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.904 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.901+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:39.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.910+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:39.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:39.961+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:49:40.027 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.024+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.046+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.060+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.110+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:49:40.165 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.162+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.181+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.194 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.190+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.242 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.239+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:49:40.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.301+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.317+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.326+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.358 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.356+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:49:40.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.397+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.411+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.422 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.419+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.449+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:49:40.501 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.498+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.516 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.513+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.524 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.521+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.563+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:49:40.653 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.651+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.690+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.707+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.753+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:49:40.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.801+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.819+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.830 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.828+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.856 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.854+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:49:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.920+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.942 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.940+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:40.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.950+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:40.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:40.987+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:49:41.033 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.031+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.046+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:41.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.054+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.082+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:49:41.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.115+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.133 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.130+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:41.141 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.138+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.168+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:49:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.235+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.271+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:41.297 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.294+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.345+0000 7f91d243bec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:49:41.444 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.441+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.478+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.499+0000 7f91d243bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:41.619 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.617+0000 7f91d243bec0 20 remove_watcher() i=0 2019-11-29T11:49:41.620 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.617+0000 7f91d243bec0 2 removed watcher, disabling cache 2019-11-29T11:49:41.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.652+0000 7f91d243bec0 20 remove_watcher() i=1 2019-11-29T11:49:41.684 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.681+0000 7f91d243bec0 20 remove_watcher() i=2 2019-11-29T11:49:41.739 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.737+0000 7f91d243bec0 20 remove_watcher() i=3 2019-11-29T11:49:41.789 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.786+0000 7f91d243bec0 20 remove_watcher() i=4 2019-11-29T11:49:41.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.837+0000 7f91d243bec0 20 remove_watcher() i=5 2019-11-29T11:49:41.891 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.884+0000 7f91d243bec0 20 remove_watcher() i=6 2019-11-29T11:49:41.938 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:41.935+0000 7f91d243bec0 20 remove_watcher() i=7 2019-11-29T11:49:41.951 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:49:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.105+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.131 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.129+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.132 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.129+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.137+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.140 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.137+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.142+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.142+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.149+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:42.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.356+0000 7fbe7b259ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:49:42.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.356+0000 7fbe7b259ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:49:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.356+0000 7fbe7b259ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:49:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.356+0000 7fbe7b259ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:49:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.356+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.375 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.373+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:42.376 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.373+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.419 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.417+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:42.420 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.417+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.434+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:42.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.434+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.446+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:42.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.446+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.456 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.453+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.456 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.453+0000 7fbe7b259ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:49:42.456 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.453+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.462 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.460+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.462 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.460+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.474+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:42.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.474+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.487+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.487+0000 7fbe7b259ec0 20 zonegroup default 2019-11-29T11:49:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.487+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.497+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.500 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.497+0000 7fbe7b259ec0 10 Cannot find current period zone using local zone 2019-11-29T11:49:42.500 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.497+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.504+0000 7fbe7b259ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:42.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.504+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.517+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:42.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.517+0000 7fbe7b259ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:42.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.537+0000 7fbe7b259ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:42.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.537+0000 7fbe7b259ec0 20 zone default found 2019-11-29T11:49:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.537+0000 7fbe7b259ec0 20 started sync module instance, tier type = 2019-11-29T11:49:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.537+0000 7fbe7b259ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:49:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=0 2019-11-29T11:49:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=1 2019-11-29T11:49:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=2 2019-11-29T11:49:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=3 2019-11-29T11:49:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=4 2019-11-29T11:49:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.956+0000 7fbe7b259ec0 20 add_watcher() i=5 2019-11-29T11:49:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.979+0000 7fbe7b259ec0 20 add_watcher() i=6 2019-11-29T11:49:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.989+0000 7fbe7b259ec0 20 add_watcher() i=7 2019-11-29T11:49:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.989+0000 7fbe7b259ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:49:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.989+0000 7fbe38fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:49:42.997 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:42.995+0000 7fbe68c87700 0 -- 172.21.15.69:0/1108661567 >> v1:172.21.15.69:6789/0 conn(0x5577b96973f0 legacy=0x5577b96ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:49:43.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:43.031+0000 7fbe67c85700 0 -- 172.21.15.69:0/2109327579 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x5577b96c2fa0 msgr2=0x5577b96c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T11:49:44.609 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.606+0000 7fbdaaffd700 20 reqs_thread_entry: start 2019-11-29T11:49:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.607+0000 7fbda8ff9700 20 reqs_thread_entry: start 2019-11-29T11:49:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.607+0000 7fbe7b259ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:49:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.607+0000 7fbd9effd700 20 reqs_thread_entry: start 2019-11-29T11:49:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.607+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:49:44.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.712+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:44.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.756+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.784+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:44.828 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.825+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:49:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.898+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.923+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.935+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:44.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:44.980+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:49:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.020+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.039+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.056 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.053+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.082+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:49:45.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.134+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.157 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.154+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.163+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.193 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.190+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:49:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.258+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.273+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.281+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.344 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.341+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:49:45.391 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.389+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.403+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.411+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.444+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:49:45.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.486+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.514+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.522+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.555 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.553+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:49:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.594+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.612+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.623+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.656 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.654+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:49:45.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.692+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.708+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.735 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.733+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.768+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:49:45.817 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.814+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.834+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:45.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.842+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:45.919 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.916+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:49:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:45.980+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.007 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.004+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.015+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.061+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:49:46.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.122+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.143+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.156+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.203 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.200+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:49:46.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.274+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.303+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.318 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.316+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.369+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:49:46.432 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.429+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.449 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.446+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.457+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.489+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:49:46.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.525+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.543 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.541+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.551+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.590+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:49:46.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.722+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.745+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.756+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.799+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:49:46.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.849+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.869 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.866+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:46.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.875+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.909+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:49:46.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.958+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:46.976+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.007 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.004+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.069 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.067+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:49:47.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.160+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.187+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.200 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.198+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.243+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:49:47.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.299+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.319+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.330+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.363 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.361+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:49:47.405 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.402+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.423+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.434+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.474 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.471+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:49:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.521+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.541+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.554 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.552+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.587+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:49:47.633 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.631+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.648 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.646+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.658 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.655+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.703 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.700+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:49:47.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.768+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.800+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.816 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.813+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.861+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:49:47.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.902+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.920+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:47.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.928+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:47.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:47.963+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:49:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.017+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.040 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.037+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.051 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.047+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.083+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:49:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.145+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.168+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.180+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.224+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:49:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.271+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.289+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.300+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.331+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:49:48.373 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.370+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.389 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.386+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.398 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.395+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.425+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:49:48.466 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.463+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.478+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.486+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.513 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.510+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:49:48.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.554+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.572+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.580+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.623+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:49:48.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.692+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.723 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.720+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:48.735 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.733+0000 7fbe7b259ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:48.820 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.817+0000 7fbe7b259ec0 20 remove_watcher() i=0 2019-11-29T11:49:48.820 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.817+0000 7fbe7b259ec0 2 removed watcher, disabling cache 2019-11-29T11:49:48.866 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.863+0000 7fbe7b259ec0 20 remove_watcher() i=1 2019-11-29T11:49:48.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.903+0000 7fbe7b259ec0 20 remove_watcher() i=2 2019-11-29T11:49:48.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:48.959+0000 7fbe7b259ec0 20 remove_watcher() i=3 2019-11-29T11:49:49.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.006+0000 7fbe7b259ec0 20 remove_watcher() i=4 2019-11-29T11:49:49.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.059+0000 7fbe7b259ec0 20 remove_watcher() i=5 2019-11-29T11:49:49.123 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.120+0000 7fbe7b259ec0 20 remove_watcher() i=6 2019-11-29T11:49:49.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.166+0000 7fbe7b259ec0 20 remove_watcher() i=7 2019-11-29T11:49:49.177 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:49:49.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.298+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.316+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.319 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.316+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.327 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.324+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.327 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.324+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.328+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.328+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.338 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.336+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:49.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.753+0000 7efd65de1ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:49:49.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.753+0000 7efd65de1ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:49:49.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.753+0000 7efd65de1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:49:49.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.753+0000 7efd65de1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:49:49.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.753+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.767+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:49.770 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.767+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.792 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.789+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:49.792 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.789+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.803+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:49.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.803+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.822+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:49.825 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.822+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.829+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.829+0000 7efd65de1ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:49:49.832 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.829+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.837+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.839 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.837+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.846+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:49.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.846+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.862+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.862+0000 7efd65de1ec0 20 zonegroup default 2019-11-29T11:49:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.862+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.871 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.868+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.871 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.868+0000 7efd65de1ec0 10 Cannot find current period zone using local zone 2019-11-29T11:49:49.871 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.868+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.874+0000 7efd65de1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:49.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.874+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.883+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:49.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.883+0000 7efd65de1ec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:49.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.909+0000 7efd65de1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:49.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.909+0000 7efd65de1ec0 20 zone default found 2019-11-29T11:49:49.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.909+0000 7efd65de1ec0 20 started sync module instance, tier type = 2019-11-29T11:49:49.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:49.909+0000 7efd65de1ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:49:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.352+0000 7efd65de1ec0 20 add_watcher() i=0 2019-11-29T11:49:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.352+0000 7efd65de1ec0 20 add_watcher() i=1 2019-11-29T11:49:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.352+0000 7efd65de1ec0 20 add_watcher() i=2 2019-11-29T11:49:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.352+0000 7efd65de1ec0 20 add_watcher() i=3 2019-11-29T11:49:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.352+0000 7efd65de1ec0 20 add_watcher() i=4 2019-11-29T11:49:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.353+0000 7efd65de1ec0 20 add_watcher() i=5 2019-11-29T11:49:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.353+0000 7efd65de1ec0 20 add_watcher() i=6 2019-11-29T11:49:50.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.387+0000 7efd65de1ec0 20 add_watcher() i=7 2019-11-29T11:49:50.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.387+0000 7efd65de1ec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:49:50.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.387+0000 7efd23fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:49:50.397 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.395+0000 7efd5380f700 0 -- 172.21.15.69:0/632798681 >> v2:172.21.15.69:3300/0 conn(0x55d9136ab450 msgr2=0x55d9136aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T11:49:50.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:50.438+0000 7efd5380f700 0 -- 172.21.15.69:0/2386496870 >> v1:172.21.15.69:6789/0 conn(0x55d9136928b0 legacy=0x55d9136d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:49:52.009 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.007+0000 7efc927fc700 20 reqs_thread_entry: start 2019-11-29T11:49:52.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.007+0000 7efc8bfff700 20 reqs_thread_entry: start 2019-11-29T11:49:52.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.007+0000 7efd65de1ec0 20 init_complete bucket index max shards: 0 2019-11-29T11:49:52.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.008+0000 7efc8a7fc700 20 reqs_thread_entry: start 2019-11-29T11:49:52.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.008+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:49:52.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.094+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.124+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.140+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.202+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:49:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.273+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.296 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.294+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.307 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.305+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.353 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.350+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:49:52.420 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.418+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.444 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.439+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.452+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.496+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:49:52.551 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.548+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.570+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.587+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.623+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:49:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.675+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.693+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.706 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.703+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.736+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:49:52.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.777+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.796 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.794+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.801+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.832+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:49:52.910 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.908+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.943+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:52.959 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:52.957+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.020+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:49:53.093 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.090+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.113+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.125+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.167+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:49:53.250 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.248+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.282+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.295+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.364 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.361+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:49:53.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.426+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.449+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.464 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.462+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.497+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:49:53.538 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.535+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.550+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.563 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.561+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.594 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.591+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:49:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.654+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.679 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.676+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.689 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.687+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.747 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.744+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:49:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.817+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.836 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.834+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.845 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.843+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.886+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:49:53.953 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.950+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:53.986 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.984+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:53.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:53.996+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.046 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.043+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:49:54.088 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.085+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.104 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.101+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.109+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.137+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:49:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.180+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.195+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.204+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.234+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:49:54.329 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.327+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.354 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.351+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.367+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.419 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.416+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:49:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.488+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.515+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.529+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.668+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:49:54.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.748+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.769+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.778+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.811+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:49:54.898 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.895+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.929+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:54.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:54.948+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.008 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.005+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:49:55.075 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.072+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.098+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.121 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.118+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.167+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:49:55.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.215+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.235+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.246 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.244+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.274+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:49:55.364 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.362+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.400+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.415+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.459+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:49:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.516+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.545 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.543+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.554 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.552+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.595 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.593+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:49:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.667+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.686+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.694+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.743+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:49:55.828 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.825+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.864 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.862+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:55.881 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.878+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:55.930 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:55.927+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:49:56.009 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.006+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.031+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.044+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.078+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:49:56.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.131+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.147+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.158 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.155+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.205+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:49:56.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.280+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.317 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.315+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.337+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.382+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:49:56.438 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.436+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.464+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.475+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.530+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:49:56.618 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.616+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.652+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.674+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.745+0000 7efd65de1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:49:56.806 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.804+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.820+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:49:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.830+0000 7efd65de1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:49:56.944 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.942+0000 7efd65de1ec0 20 remove_watcher() i=0 2019-11-29T11:49:56.945 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.942+0000 7efd65de1ec0 2 removed watcher, disabling cache 2019-11-29T11:49:56.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:56.983+0000 7efd65de1ec0 20 remove_watcher() i=1 2019-11-29T11:49:57.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.026+0000 7efd65de1ec0 20 remove_watcher() i=2 2019-11-29T11:49:57.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.105+0000 7efd65de1ec0 20 remove_watcher() i=3 2019-11-29T11:49:57.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.145+0000 7efd65de1ec0 20 remove_watcher() i=4 2019-11-29T11:49:57.198 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.195+0000 7efd65de1ec0 20 remove_watcher() i=5 2019-11-29T11:49:57.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.252+0000 7efd65de1ec0 20 remove_watcher() i=6 2019-11-29T11:49:57.313 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.310+0000 7efd65de1ec0 20 remove_watcher() i=7 2019-11-29T11:49:57.323 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T11:49:57.493 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.491+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:57.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.517+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:57.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.517+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:57.739 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.736+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:57.740 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.736+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:57.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.746+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:57.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.746+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:57.764 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:57.760+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.001+0000 7f104cf5aec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T11:49:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.001+0000 7f104cf5aec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T11:49:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.001+0000 7f104cf5aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T11:49:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.001+0000 7f104cf5aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T11:49:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.001+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.016+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:58.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.016+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.035 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.033+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:58.035 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.033+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.043+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.043+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.059+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.059+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.066+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.066+0000 7f104cf5aec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T11:49:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.066+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.075+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.075+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.090+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.090+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.104+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T11:49:58.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.104+0000 7f104cf5aec0 20 zonegroup default 2019-11-29T11:49:58.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.104+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.110+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:58.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.110+0000 7f104cf5aec0 10 Cannot find current period zone using local zone 2019-11-29T11:49:58.112 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.110+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.115+0000 7f104cf5aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T11:49:58.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.115+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.127+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T11:49:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.127+0000 7f104cf5aec0 20 rados->read ofs=0 len=0 2019-11-29T11:49:58.149 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.147+0000 7f104cf5aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T11:49:58.149 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.147+0000 7f104cf5aec0 20 zone default found 2019-11-29T11:49:58.150 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.147+0000 7f104cf5aec0 20 started sync module instance, tier type = 2019-11-29T11:49:58.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.147+0000 7f104cf5aec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T11:49:58.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=0 2019-11-29T11:49:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=1 2019-11-29T11:49:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=2 2019-11-29T11:49:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=3 2019-11-29T11:49:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=4 2019-11-29T11:49:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.587+0000 7f104cf5aec0 20 add_watcher() i=5 2019-11-29T11:49:58.619 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.617+0000 7f104cf5aec0 20 add_watcher() i=6 2019-11-29T11:49:58.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.648+0000 7f103a187700 0 -- 172.21.15.69:0/3152247137 >> [v2:172.21.15.163:6816/9735,v1:172.21.15.163:6817/9735] conn(0x7f101c028830 msgr2=0x7f101c02afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:49:58.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.648+0000 7f104cf5aec0 20 add_watcher() i=7 2019-11-29T11:49:58.650 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.648+0000 7f104cf5aec0 2 all 8 watchers are set, enabling cache 2019-11-29T11:49:58.651 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.648+0000 7f100afdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T11:49:58.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:49:58.675+0000 7f103a988700 0 -- 172.21.15.69:0/3362303962 >> v2:172.21.15.69:3300/0 conn(0x55e5ba1c3df0 msgr2=0x55e5ba1c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T11:50:00.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.094+0000 7f0f797fa700 20 reqs_thread_entry: start 2019-11-29T11:50:00.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.094+0000 7f0f72ffd700 20 reqs_thread_entry: start 2019-11-29T11:50:00.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.095+0000 7f104cf5aec0 20 init_complete bucket index max shards: 0 2019-11-29T11:50:00.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.095+0000 7f0f717fa700 20 reqs_thread_entry: start 2019-11-29T11:50:00.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.095+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T11:50:00.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.137+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.154 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.152+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.161+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.207+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T11:50:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.243+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.261+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.271 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.269+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.299 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.297+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T11:50:00.424 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.422+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.446+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.458+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.507 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.505+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T11:50:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.563+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.592+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.608 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.606+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.652 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.650+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T11:50:00.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.701+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.723+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.734+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.771+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T11:50:00.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.821+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.838+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:00.849 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.847+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:00.882 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.881+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T11:50:00.979 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:00.977+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.009+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.017+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.052 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.050+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T11:50:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.109+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.128+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.140 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.138+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.182 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.180+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T11:50:01.236 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.234+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.251 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.250+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.258 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.257+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.293+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T11:50:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.340+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.379+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.388+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.422 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.420+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T11:50:01.458 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.457+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.482 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.480+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.495 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.493+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.523+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T11:50:01.569 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.567+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.584+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.593+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.645 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.643+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T11:50:01.656 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:37:14.546 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.663 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:37:14.553 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.667 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:37:14.557 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.673 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:37:14.449 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.673 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:37:14.563 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.679 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:37:14.455 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.692 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.690+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.716 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.714+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.725+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.766+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T11:50:01.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.813+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.820 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:37:14.710 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.833 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:37:14.723 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.838+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.850 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.848+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.860 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:37:14.635 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:50:01.880 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.879+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T11:50:01.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.923+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:01.949 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.947+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:01.963+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.002+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T11:50:02.052 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.051+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.072 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.070+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.082+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.122 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.120+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T11:50:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.171+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.190+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.202 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.200+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.239 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.237+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T11:50:02.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.279+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.295+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.306 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.304+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.335 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.333+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T11:50:02.409 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.408+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.445+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.457+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.503 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.501+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T11:50:02.566 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.564+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.584 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.582+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.595+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.629+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T11:50:02.698 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.697+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.712+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.723 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.721+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.765 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.764+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T11:50:02.812 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.811+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.831 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.829+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.840 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.838+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.872+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T11:50:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.929+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.954+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:02.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.961+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:02.997+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T11:50:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.046+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.061+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.072 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.070+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.105+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T11:50:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.175+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.203+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.213+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.255 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.253+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T11:50:03.343 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.341+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.367+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.377+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.414+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T11:50:03.465 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.464+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.482+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.490+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.531 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.530+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T11:50:03.586 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.584+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.626 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.624+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.644 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.642+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.669+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T11:50:03.776 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.774+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.804+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.817+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.869 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.867+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T11:50:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.925+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.943+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:03.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.953+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:03.995 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:03.994+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T11:50:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.042+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:04.060 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.058+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:04.070 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.068+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.109+0000 7f104cf5aec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T11:50:04.149 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.147+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:04.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.166+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T11:50:04.183 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.182+0000 7f104cf5aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T11:50:04.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.266+0000 7f104cf5aec0 20 remove_watcher() i=0 2019-11-29T11:50:04.268 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.266+0000 7f104cf5aec0 2 removed watcher, disabling cache 2019-11-29T11:50:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.294+0000 7f104cf5aec0 20 remove_watcher() i=1 2019-11-29T11:50:04.329 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.328+0000 7f104cf5aec0 20 remove_watcher() i=2 2019-11-29T11:50:04.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.398+0000 7f104cf5aec0 20 remove_watcher() i=3 2019-11-29T11:50:04.451 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.449+0000 7f104cf5aec0 20 remove_watcher() i=4 2019-11-29T11:50:04.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.518+0000 7f104cf5aec0 20 remove_watcher() i=5 2019-11-29T11:50:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.560+0000 7f104cf5aec0 20 remove_watcher() i=6 2019-11-29T11:50:04.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T11:50:04.601+0000 7f104cf5aec0 20 remove_watcher() i=7 2019-11-29T11:50:04.615 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-11-29T11:50:05.168 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :clean 2019-11-29T11:50:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileJava 2019-11-29T11:50:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processResources 2019-11-29T11:50:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :classes 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :compileTestJava 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :processTestResources NO-SOURCE 2019-11-29T11:50:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :testClasses 2019-11-29T11:50:07.965 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:07.965 INFO:teuthology.orchestra.run.smithi069.stdout:> Task :test 2019-11-29T11:50:07.965 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:07.965 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-11-29T11:50:13.565 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:13.565 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-11-29T11:50:25.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:25.265 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-11-29T11:50:28.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:28.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-11-29T11:50:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:35.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-11-29T11:50:37.565 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:37.565 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-11-29T11:50:44.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:50:44.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-11-29T11:51:09.765 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:09.765 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-11-29T11:51:17.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:17.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-11-29T11:51:20.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:20.665 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-11-29T11:51:25.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:25.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-11-29T11:51:30.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:30.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-11-29T11:51:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:38.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-11-29T11:51:38.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:38.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-11-29T11:51:39.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:39.965 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-11-29T11:51:42.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:42.865 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-11-29T11:51:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:48.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-11-29T11:51:55.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:51:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-11-29T11:52:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:03.566 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-11-29T11:52:07.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:07.065 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-11-29T11:52:15.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:15.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-11-29T11:52:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:18.765 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-11-29T11:52:26.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:26.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-11-29T11:52:28.465 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:28.465 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-11-29T11:52:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:34.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-11-29T11:52:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:36.565 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-11-29T11:52:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:41.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-11-29T11:52:43.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:43.465 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-11-29T11:52:49.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:49.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-11-29T11:52:51.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:51.765 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-11-29T11:52:57.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:52:57.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-11-29T11:53:02.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:53:02.065 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-11-29T11:53:09.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:53:09.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-11-29T11:53:26.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:53:26.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-11-29T11:53:40.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:53:40.365 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-11-29T11:55:12.907 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:25.798 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:12.913 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:25.804 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:12.928 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:25.818 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:12.929 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:25.705 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:12.933 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:25.823 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:12.934 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:25.709 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:13.112 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:26.002 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:13.116 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:26.007 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:13.233 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:26.008 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.923 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:32.813 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.933 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:32.822 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.961 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:42:32.851 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.964 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:32.854 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.969 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:32.859 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.986 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:32.761 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.987 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:42:32.876 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:19.990 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:32.766 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:55:20.015 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:42:32.790 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T11:57:20.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:57:20.765 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-11-29T11:57:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:57:35.265 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-11-29T11:57:38.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:57:38.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-11-29T11:57:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:57:44.865 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-11-29T11:57:50.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:57:50.965 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-11-29T11:58:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:05.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-11-29T11:58:16.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:16.665 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-11-29T11:58:23.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:23.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-11-29T11:58:24.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:24.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-11-29T11:58:26.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:26.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-11-29T11:58:43.654 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T11:58:43.601+0000 41a62700 -1 handle_data failed with exit code -5 2019-11-29T11:58:43.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:43.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-11-29T11:58:51.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:58:51.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-11-29T11:59:02.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:59:02.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-11-29T11:59:15.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:59:15.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-11-29T11:59:39.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:59:39.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-11-29T11:59:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T11:59:46.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-11-29T12:00:23.126 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:47:36.017 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.135 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:47:36.026 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.149 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:47:36.039 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.154 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:47:36.044 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.163 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:47:35.938 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.168 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:47:35.943 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.314 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:47:36.204 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.376 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:47:36.265 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:00:23.422 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:47:36.197 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:02:16.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:02:16.665 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-11-29T12:02:24.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:02:24.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-11-29T12:02:52.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:02:52.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-11-29T12:03:00.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:03:00.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-11-29T12:03:28.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:03:28.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-11-29T12:03:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:03:36.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-11-29T12:03:42.470 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:03:42.470 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-11-29T12:03:46.350 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:50:59.240 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.364 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:50:59.255 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.370 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:50:59.260 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.374 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:50:59.264 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.378 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:50:59.152 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.404 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:50:59.179 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.469 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:50:59.358 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.471 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:50:59.362 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:03:46.498 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:50:59.272 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:04:08.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:08.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-11-29T12:04:09.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:09.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-11-29T12:04:10.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:10.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-11-29T12:04:11.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:11.165 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-11-29T12:04:14.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:14.165 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-11-29T12:04:16.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-11-29T12:04:20.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:20.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-11-29T12:04:21.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:21.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-11-29T12:04:25.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-11-29T12:04:26.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:26.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-11-29T12:04:30.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:30.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-11-29T12:04:31.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:31.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-11-29T12:04:37.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:37.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-11-29T12:04:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:41.165 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-11-29T12:04:45.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:45.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-11-29T12:04:49.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:49.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-11-29T12:04:56.366 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:04:56.366 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-11-29T12:05:02.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:02.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-11-29T12:05:12.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:12.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-11-29T12:05:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-11-29T12:05:29.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-11-29T12:05:33.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:33.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-11-29T12:05:36.158 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:52:49.048 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.167 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:52:49.058 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.174 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:52:49.064 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.176 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:52:49.066 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.184 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:52:48.958 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.191 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:52:48.965 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.245 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:52:49.136 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.251 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:52:49.141 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:36.291 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:52:49.065 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:05:42.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:42.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-11-29T12:05:48.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:48.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-11-29T12:05:56.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:05:56.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-11-29T12:06:03.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-11-29T12:06:14.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:14.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-11-29T12:06:21.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:21.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-11-29T12:06:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:35.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-11-29T12:06:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:43.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-11-29T12:06:56.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:06:56.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-11-29T12:07:06.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:07:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-11-29T12:07:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:07:19.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-11-29T12:07:26.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:07:26.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-11-29T12:07:39.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:07:39.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-11-29T12:07:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:07:48.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-11-29T12:08:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:00.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-11-29T12:08:09.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:09.165 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-11-29T12:08:24.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:24.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-11-29T12:08:32.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:32.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-11-29T12:08:42.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:42.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-11-29T12:08:49.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:08:49.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-11-29T12:09:01.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:01.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-11-29T12:09:09.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:09.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-11-29T12:09:23.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:23.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-11-29T12:09:31.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:31.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-11-29T12:09:45.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-11-29T12:09:52.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:09:52.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-11-29T12:10:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:04.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-11-29T12:10:12.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:12.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-11-29T12:10:24.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:24.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-11-29T12:10:33.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:33.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-11-29T12:10:46.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:46.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-11-29T12:10:48.661 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:58:01.551 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.672 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:58:01.563 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.684 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:58:01.574 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.689 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:58:01.579 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.695 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:58:01.468 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.705 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:58:01.478 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.882 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:58:01.771 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.947 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:58:01.721 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:48.949 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:58:01.840 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:10:51.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:10:51.164 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-11-29T12:11:00.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:00.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-11-29T12:11:06.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:06.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-11-29T12:11:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:18.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-11-29T12:11:23.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:23.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-11-29T12:11:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:35.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-11-29T12:11:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:41.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-11-29T12:11:52.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:52.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-11-29T12:11:55.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:11:55.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-11-29T12:12:02.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:02.364 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-11-29T12:12:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:04.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-11-29T12:12:07.818 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:59:20.709 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.822 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:59:20.713 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.866 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:59:20.756 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.867 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:59:20.757 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.896 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:00:59:20.787 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.905 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:59:20.679 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.917 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:00:59:20.807 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:07.920 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:59:20.694 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:08.054 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:00:59:20.828 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:12:09.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:09.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-11-29T12:12:13.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:13.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-11-29T12:12:23.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:23.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-11-29T12:12:27.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:27.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-11-29T12:12:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:35.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-11-29T12:12:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:42.165 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-11-29T12:12:49.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:49.964 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-11-29T12:12:50.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:50.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-11-29T12:12:51.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:51.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-11-29T12:12:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:12:57.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-11-29T12:13:06.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:06.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-11-29T12:13:11.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:11.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-11-29T12:13:21.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:21.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-11-29T12:13:24.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:24.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-11-29T12:13:33.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:33.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-11-29T12:13:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:35.564 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-11-29T12:13:43.864 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:43.864 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-11-29T12:13:48.264 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:48.264 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-11-29T12:13:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:13:58.064 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-11-29T12:14:10.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:14:10.764 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-11-29T12:14:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:14:18.464 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-11-29T12:14:31.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:14:31.664 INFO:teuthology.orchestra.run.smithi069.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-11-29T12:14:40.764 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout:---------------------------------------------------------------------- 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout:---------------------------------------------------------------------- 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout:BUILD SUCCESSFUL in 24m 36s 2019-11-29T12:14:40.765 INFO:teuthology.orchestra.run.smithi069.stdout:5 actionable tasks: 5 executed 2019-11-29T12:14:40.779 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T12:14:40.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.913+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.931+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.931+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:40.940 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.939+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:40.940 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.939+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:40.944 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.943+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:40.945 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.943+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:40.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:40.951+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:14:41.158 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.157+0000 7f8ec90e9ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:14:41.158 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.157+0000 7f8ec90e9ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:14:41.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.157+0000 7f8ec90e9ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:14:41.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.157+0000 7f8ec90e9ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:14:41.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.157+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.172+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:14:41.173 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.172+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.193 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.192+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:14:41.193 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.192+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.204+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:14:41.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.204+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.220+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:14:41.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.220+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.226+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:41.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.226+0000 7f8ec90e9ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:14:41.228 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.226+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.237+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.237+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.252+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:14:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.252+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.269+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:14:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.269+0000 7f8ec90e9ec0 20 zonegroup default 2019-11-29T12:14:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.269+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.275+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:41.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.275+0000 7f8ec90e9ec0 10 Cannot find current period zone using local zone 2019-11-29T12:14:41.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.275+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.283+0000 7f8ec90e9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:14:41.285 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.283+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.298 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.296+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:14:41.298 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.296+0000 7f8ec90e9ec0 20 rados->read ofs=0 len=0 2019-11-29T12:14:41.314 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.311+0000 7f8ec90e9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:14:41.315 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.311+0000 7f8ec90e9ec0 20 zone default found 2019-11-29T12:14:41.315 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.311+0000 7f8ec90e9ec0 20 started sync module instance, tier type = 2019-11-29T12:14:41.316 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.311+0000 7f8ec90e9ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:14:41.701 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=0 2019-11-29T12:14:41.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=1 2019-11-29T12:14:41.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=2 2019-11-29T12:14:41.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=3 2019-11-29T12:14:41.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=4 2019-11-29T12:14:41.702 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.700+0000 7f8ec90e9ec0 20 add_watcher() i=5 2019-11-29T12:14:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.727+0000 7f8ec90e9ec0 20 add_watcher() i=6 2019-11-29T12:14:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.737+0000 7f8ec90e9ec0 20 add_watcher() i=7 2019-11-29T12:14:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.737+0000 7f8ec90e9ec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:14:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.737+0000 7f8e86fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:14:41.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.744+0000 7f8eb6316700 0 -- 172.21.15.69:0/2207068960 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a1290cea40 msgr2=0x55a1290f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T12:14:41.947 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.946+0000 7f8eb6316700 0 --2- 172.21.15.69:0/2207068960 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a1290cea40 0x55a1290f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-29T12:14:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:41.997+0000 7f8eb6b17700 0 -- 172.21.15.69:0/726321170 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a1290e3380 msgr2=0x55a1290caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:14:43.632 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.630+0000 7f8df57fa700 20 reqs_thread_entry: start 2019-11-29T12:14:43.632 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.631+0000 7f8deeffd700 20 reqs_thread_entry: start 2019-11-29T12:14:43.632 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.631+0000 7f8ec90e9ec0 20 init_complete bucket index max shards: 0 2019-11-29T12:14:43.633 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.631+0000 7f8ded7fa700 20 reqs_thread_entry: start 2019-11-29T12:14:43.633 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.631+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T12:14:43.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.716+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:43.754 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.753+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:43.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.781+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.836+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T12:14:43.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.935+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:43.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.963+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:43.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:14:43.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ', time=2019-11-29T13:51:09.100124+0000, chain.objs.size()=9 2019-11-29T12:14:43.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__multipart_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.1 2019-11-29T12:14:43.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__shadow_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.1_1 2019-11-29T12:14:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__multipart_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.2 2019-11-29T12:14:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__shadow_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.2_1 2019-11-29T12:14:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__multipart_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.3 2019-11-29T12:14:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__shadow_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.3_1 2019-11-29T12:14:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__multipart_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.4 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__shadow_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.4_1 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.52__multipart_key1.2~loTyQWCrxb8i9x59r-JWTY3HCQzmufQ.5 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1064', time=2019-11-29T13:55:29.171547+0000, chain.objs.size()=9 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.1 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.1_1 2019-11-29T12:14:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:43.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.2 2019-11-29T12:14:44.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.395+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.2_1 2019-11-29T12:14:44.468 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.467+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.3 2019-11-29T12:14:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.467+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.3_1 2019-11-29T12:14:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.467+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.4 2019-11-29T12:14:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.467+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.4_1 2019-11-29T12:14:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.467+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~8xLx5qRD13F5JZqFb6XwxlNfO_sq67v.5 2019-11-29T12:14:44.907 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:44.906+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:45.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.011+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T12:14:45.051 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.050+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:45.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.066+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1231', time=2019-11-29T14:01:18.664002+0000, chain.objs.size()=9 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.1 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.1_1 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.2 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.2_1 2019-11-29T12:14:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.3 2019-11-29T12:14:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.3_1 2019-11-29T12:14:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.4 2019-11-29T12:14:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.4_1 2019-11-29T12:14:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.080+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~SQK3t2JE9S0lycA18twP2eUeLrnh0tC.5 2019-11-29T12:14:45.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.557+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:45.877 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.876+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T12:14:45.941 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.940+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:45.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.963+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:45.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.980+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.980+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1086', time=2019-11-29T13:56:52.849630+0000, chain.objs.size()=7 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.1 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.1_1 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.1_2 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.2 2019-11-29T12:14:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.2_1 2019-11-29T12:14:45.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.2_2 2019-11-29T12:14:45.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~DwPUxuk7k4K7iHDfse0zTOSBZrP5Rbx.3 2019-11-29T12:14:45.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1135', time=2019-11-29T13:58:18.765648+0000, chain.objs.size()=9 2019-11-29T12:14:45.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__multipart_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.1 2019-11-29T12:14:45.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__shadow_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.1_1 2019-11-29T12:14:45.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__multipart_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.2 2019-11-29T12:14:45.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__shadow_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.2_1 2019-11-29T12:14:45.985 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:45.981+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__multipart_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.3 2019-11-29T12:14:46.220 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.219+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__shadow_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.3_1 2019-11-29T12:14:46.227 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.226+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__multipart_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.4 2019-11-29T12:14:46.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.283+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__shadow_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.4_1 2019-11-29T12:14:46.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.283+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.71__multipart_key1.2~xs-2_qdmIt4jy2mwOPOTi5fGeT3uFzB.5 2019-11-29T12:14:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.797+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:46.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.909+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T12:14:46.974 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.972+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:46.996+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:47.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.008+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:47.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.008+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1098', time=2019-11-29T13:57:22.880652+0000, chain.objs.size()=7 2019-11-29T12:14:47.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.008+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.1 2019-11-29T12:14:47.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.008+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.1_1 2019-11-29T12:14:47.010 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.008+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.1_2 2019-11-29T12:14:47.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.009+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.2 2019-11-29T12:14:47.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.009+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.2_1 2019-11-29T12:14:47.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.009+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.2_2 2019-11-29T12:14:47.011 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.009+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~AMOFZcvpXpH97Hu5G84ezyfAC9xncrV.3 2019-11-29T12:14:47.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.289+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:47.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.513+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T12:14:47.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.563+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.582+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:47.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.592+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:47.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.629+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T12:14:47.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.704+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:47.730 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.729+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.741+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.741+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.972', time=2019-11-29T13:51:57.609160+0000, chain.objs.size()=2 2019-11-29T12:14:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.741+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.56__shadow_.FVFfqU8pzqsfM6GgP4X8z91ulQMphRJ_1 2019-11-29T12:14:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.741+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.56__shadow_.FVFfqU8pzqsfM6GgP4X8z91ulQMphRJ_2 2019-11-29T12:14:47.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:47.954+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:48.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.082+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T12:14:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.131+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.150+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1072', time=2019-11-29T13:55:55.819067+0000, chain.objs.size()=9 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.1 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.1_1 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.2 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.2_1 2019-11-29T12:14:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.3 2019-11-29T12:14:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.3_1 2019-11-29T12:14:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.4 2019-11-29T12:14:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.4_1 2019-11-29T12:14:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.169+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Y1PfRiVFhHrsI51EqJIwTNQ2Yy8nj0x.5 2019-11-29T12:14:48.582 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.581+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:48.658 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.657+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T12:14:48.693 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.692+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:48.711 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.710+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:48.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:48.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1241', time=2019-11-29T14:02:15.128766+0000, chain.objs.size()=7 2019-11-29T12:14:48.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.1 2019-11-29T12:14:48.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.1_1 2019-11-29T12:14:48.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.1_2 2019-11-29T12:14:48.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.2 2019-11-29T12:14:48.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.2_1 2019-11-29T12:14:48.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.2_2 2019-11-29T12:14:48.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:48.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~rjcotFcgpLOPYsvWbkrRFY2_GcRR838.3 2019-11-29T12:14:49.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.075+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:49.291 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.289+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T12:14:49.350 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.349+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:49.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.368+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1092', time=2019-11-29T13:57:18.771701+0000, chain.objs.size()=7 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.1 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.1_1 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.1_2 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.2 2019-11-29T12:14:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.2_1 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.2_2 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~Kyb-iJd9plAIrP3MQPcdX8XuKjFH4H-.3 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG', time=2019-11-29T14:04:00.408234+0000, chain.objs.size()=52 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.1 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.2 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.3 2019-11-29T12:14:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.4 2019-11-29T12:14:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.404+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.5 2019-11-29T12:14:49.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.6 2019-11-29T12:14:49.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.7 2019-11-29T12:14:49.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.8 2019-11-29T12:14:49.724 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.9 2019-11-29T12:14:49.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.10 2019-11-29T12:14:49.758 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.757+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.11 2019-11-29T12:14:49.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.757+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.12 2019-11-29T12:14:49.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.757+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.13 2019-11-29T12:14:49.759 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.757+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.14 2019-11-29T12:14:49.908 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.907+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.lrrTy2vOPUqgbTpV7_wHkCUQbDMQiS8.15 2019-11-29T12:14:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:49.907+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.dAofkX-lY_BuTZH_jTiyhzd27edxhri.16 2019-11-29T12:14:50.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.106+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.31WOf8VB8F543UpsMOiie90tBELoMSn.17 2019-11-29T12:14:50.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.182+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.CkVwZ4cYLCYwsBsne2HnNuEK_xrMTR5.18 2019-11-29T12:14:50.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.182+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.YcwHHuQ0GQzT22GHHzGJ-Rg-TQ2L0ub.19 2019-11-29T12:14:50.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.182+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.dpAV31wOk4YSYxtifqgtk7vSwZ11sMl.20 2019-11-29T12:14:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.267+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.21 2019-11-29T12:14:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.267+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.22 2019-11-29T12:14:50.392 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.389+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.23 2019-11-29T12:14:50.392 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.389+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.24 2019-11-29T12:14:50.392 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.390+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.25 2019-11-29T12:14:50.570 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.568+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.26 2019-11-29T12:14:50.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.568+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.27 2019-11-29T12:14:50.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.578+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.28 2019-11-29T12:14:50.580 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:50.578+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.29 2019-11-29T12:14:51.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.30 2019-11-29T12:14:51.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.31 2019-11-29T12:14:51.453 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.32 2019-11-29T12:14:51.453 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.33 2019-11-29T12:14:51.453 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.34 2019-11-29T12:14:51.453 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.450+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.35 2019-11-29T12:14:51.504 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.501+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.36 2019-11-29T12:14:51.504 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.501+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.37 2019-11-29T12:14:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.881+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.38 2019-11-29T12:14:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.881+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.39 2019-11-29T12:14:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:51.881+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.40 2019-11-29T12:14:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.190+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.41 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.42 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.43 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.44 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.45 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.46 2019-11-29T12:14:52.459 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.47 2019-11-29T12:14:52.460 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.456+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.48 2019-11-29T12:14:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.758+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.49 2019-11-29T12:14:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.758+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.50 2019-11-29T12:14:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.758+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.51 2019-11-29T12:14:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:52.758+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.78__multipart_key1.2~BQ-6TzglwL7-F11DXHcQOe4E8e_EdXG.52 2019-11-29T12:14:53.561 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.559+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.640+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T12:14:53.697 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.695+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:53.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.710+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1261', time=2019-11-29T14:02:55.372016+0000, chain.objs.size()=9 2019-11-29T12:14:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__multipart_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.1 2019-11-29T12:14:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__shadow_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.1_1 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__multipart_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.2 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.723+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__shadow_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.2_1 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.724+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__multipart_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.3 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.724+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__shadow_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.3_1 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.724+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__multipart_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.4 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.724+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__shadow_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.4_1 2019-11-29T12:14:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:53.724+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.76__multipart_key1.2~v34UtJWU4r0GkZPGHUfH1apCH31QFxw.5 2019-11-29T12:14:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.370+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:54.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.518+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T12:14:54.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.568+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.602+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:54.616 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.614+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:54.649 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.647+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T12:14:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.735+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:54.754 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.752+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:54.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:54.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1224', time=2019-11-29T14:01:02.435425+0000, chain.objs.size()=9 2019-11-29T12:14:54.767 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.1 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.1_1 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.2 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.2_1 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.3 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.3_1 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.4 2019-11-29T12:14:54.768 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.4_1 2019-11-29T12:14:54.769 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:54.764+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~hkLq7Q09Y6ekqstYgzjHCD7l6gEJVGU.5 2019-11-29T12:14:55.451 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.448+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.565+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T12:14:55.621 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.618+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:55.642 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.640+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:14:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1080', time=2019-11-29T13:56:25.829407+0000, chain.objs.size()=9 2019-11-29T12:14:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.1 2019-11-29T12:14:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.1_1 2019-11-29T12:14:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.2 2019-11-29T12:14:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.2_1 2019-11-29T12:14:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.3 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.3_1 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.4 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__shadow_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.4_1 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.67__multipart_key1.2~SkPRF6KscuPt9Hu0AxIkWihTJGJb-ji.5 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1236', time=2019-11-29T14:01:39.621728+0000, chain.objs.size()=7 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.1 2019-11-29T12:14:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.1_1 2019-11-29T12:14:55.664 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.658+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.1_2 2019-11-29T12:14:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.999+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.2 2019-11-29T12:14:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.999+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.2_1 2019-11-29T12:14:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.999+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.2_2 2019-11-29T12:14:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:55.999+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~A9k8WMSXghBDpjJ9ldQ1RuGAnPpnCDu.3 2019-11-29T12:14:56.440 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.437+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:56.591 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.588+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T12:14:56.636 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.634+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:56.656 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.652+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1246', time=2019-11-29T14:02:19.144429+0000, chain.objs.size()=7 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.1 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.1_1 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.1_2 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.2 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.2_1 2019-11-29T12:14:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.2_2 2019-11-29T12:14:56.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:56.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~R2qSW53RqewA1i6I0fx3YifmkQNvXdo.3 2019-11-29T12:14:57.003 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.001+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.241+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T12:14:57.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.289+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.313+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.335+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:14:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.335+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1043', time=2019-11-29T13:53:28.692349+0000, chain.objs.size()=9 2019-11-29T12:14:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.335+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.1 2019-11-29T12:14:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.1_1 2019-11-29T12:14:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.2 2019-11-29T12:14:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.2_1 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.3 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.3_1 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.4 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.4_1 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.5 2019-11-29T12:14:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1041', time=2019-11-29T13:53:34.940353+0000, chain.objs.size()=9 2019-11-29T12:14:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.1 2019-11-29T12:14:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.1_1 2019-11-29T12:14:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.336+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.2 2019-11-29T12:14:57.570 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.567+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.2_1 2019-11-29T12:14:57.570 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.567+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.3 2019-11-29T12:14:57.570 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.567+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.3_1 2019-11-29T12:14:57.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.574+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.4 2019-11-29T12:14:57.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.574+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__shadow_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.4_1 2019-11-29T12:14:57.576 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.574+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.64__multipart_key1.2~f2iyhDjx7iOzDf7aHPSFcwkvOxv7Y5d.5 2019-11-29T12:14:57.999 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:57.997+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:58.126 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.124+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T12:14:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.353+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:58.383 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.381+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.400+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:58.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.440+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T12:14:58.497 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.494+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.531+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:58.553 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.551+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.585+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T12:14:58.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.637+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:58.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.653+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1185', time=2019-11-29T13:59:07.495399+0000, chain.objs.size()=9 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.1 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.1_1 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.2 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.2_1 2019-11-29T12:14:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.3 2019-11-29T12:14:58.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.3_1 2019-11-29T12:14:58.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.4 2019-11-29T12:14:58.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.4_1 2019-11-29T12:14:58.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:58.668+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file2.mpg.2~7JuLEYwPAsuZvgq791fgXEaLreTPPcc.5 2019-11-29T12:14:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.097+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:14:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.357+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T12:14:59.410 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.408+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:59.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.427+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:59.439 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.437+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.480+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T12:14:59.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.538+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:14:59.555 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.553+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:14:59.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:14:59.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK', time=2019-11-29T13:59:42.688027+0000, chain.objs.size()=9 2019-11-29T12:14:59.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__multipart_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.7 2019-11-29T12:14:59.567 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__shadow_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.7_1 2019-11-29T12:14:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__multipart_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.8 2019-11-29T12:14:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__shadow_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.8_1 2019-11-29T12:14:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__multipart_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.9 2019-11-29T12:14:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__shadow_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.9_1 2019-11-29T12:14:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__multipart_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.10 2019-11-29T12:14:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__shadow_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.10_1 2019-11-29T12:14:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:14:59.565+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.74__multipart_key1.2~ZqxznOVnn5Vo4We1eE0JqOCuaobUAKK.11 2019-11-29T12:15:00.083 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.081+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:00.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.216+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T12:15:00.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.300+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:00.320 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.318+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:00.358 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1907', time=2019-11-29T14:14:13.109111+0000, chain.objs.size()=9 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__multipart_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.1 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__shadow_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.1_1 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__multipart_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.2 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__shadow_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.2_1 2019-11-29T12:15:00.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__multipart_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.3 2019-11-29T12:15:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__shadow_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.3_1 2019-11-29T12:15:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__multipart_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.4 2019-11-29T12:15:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__shadow_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.4_1 2019-11-29T12:15:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.356+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.118__multipart_key1.2~6zhJa8F-fnmiAdI5XIKswZ4nGqAWWlA.5 2019-11-29T12:15:00.807 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.805+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:00.900 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.898+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T12:15:00.957 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.955+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:00.973 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.971+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:00.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-29T12:15:00.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1217', time=2019-11-29T14:00:38.584020+0000, chain.objs.size()=9 2019-11-29T12:15:00.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.1 2019-11-29T12:15:00.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.1_1 2019-11-29T12:15:00.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.2 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.2_1 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.3 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.3_1 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.4 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__shadow_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.4_1 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.75__multipart_key1.2~yiFRtCbVVoVp_rttBO--LOWsTeOyfzI.5 2019-11-29T12:15:00.991 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1276', time=2019-11-29T14:03:30.079971+0000, chain.objs.size()=9 2019-11-29T12:15:00.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__multipart_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.1 2019-11-29T12:15:00.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__shadow_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.1_1 2019-11-29T12:15:00.992 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:00.987+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__multipart_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.2 2019-11-29T12:15:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.326+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__shadow_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.2_1 2019-11-29T12:15:01.329 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.326+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__multipart_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.3 2019-11-29T12:15:01.329 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.326+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__shadow_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.3_1 2019-11-29T12:15:01.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.425+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__multipart_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.4 2019-11-29T12:15:01.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.425+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__shadow_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.4_1 2019-11-29T12:15:01.554 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.552+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.77__multipart_key1.2~2d_ebVR55AJ_SmnNCpkAKSrEG0ghqm4.5 2019-11-29T12:15:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.896+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:01.984 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:01.978+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T12:15:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.016+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.032+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.043+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.078+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T12:15:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.189+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.223 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.221+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:02.239 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.237+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.282+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T12:15:02.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.330+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.348+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:02.363 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.360+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.392+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T12:15:02.485 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.483+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:02.516 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.514+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:15:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1154', time=2019-11-29T13:58:46.454573+0000, chain.objs.size()=9 2019-11-29T12:15:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__multipart_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.1 2019-11-29T12:15:02.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__shadow_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.1_1 2019-11-29T12:15:02.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__multipart_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.2 2019-11-29T12:15:02.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.531+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__shadow_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.2_1 2019-11-29T12:15:02.535 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.532+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__multipart_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.3 2019-11-29T12:15:02.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.532+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__shadow_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.3_1 2019-11-29T12:15:02.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.532+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__multipart_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.4 2019-11-29T12:15:02.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.532+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__shadow_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.4_1 2019-11-29T12:15:02.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.532+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.72__multipart_key1.2~3CQnfXTl8h765s7qVMQj8Phg8bDKCl5.5 2019-11-29T12:15:02.973 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:02.971+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.327+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T12:15:03.385 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.382+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:03.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.399+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:03.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:15:03.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1056', time=2019-11-29T13:57:28.990791+0000, chain.objs.size()=5 2019-11-29T12:15:03.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.66__shadow_.i5zK7mBKHTgZhE6z0TJQFz3FG8VU8hM_1 2019-11-29T12:15:03.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.66__shadow_.i5zK7mBKHTgZhE6z0TJQFz3FG8VU8hM_2 2019-11-29T12:15:03.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.66__shadow_.i5zK7mBKHTgZhE6z0TJQFz3FG8VU8hM_3 2019-11-29T12:15:03.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.66__shadow_.i5zK7mBKHTgZhE6z0TJQFz3FG8VU8hM_4 2019-11-29T12:15:03.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.411+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.66__shadow_.i5zK7mBKHTgZhE6z0TJQFz3FG8VU8hM_5 2019-11-29T12:15:03.697 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.695+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:03.738 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.736+0000 7f8e86fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:03.763 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.761+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T12:15:03.801 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.799+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:03.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.821+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:03.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:15:03.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1923', time=2019-11-29T14:14:33.861496+0000, chain.objs.size()=9 2019-11-29T12:15:03.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__multipart_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.1 2019-11-29T12:15:03.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__shadow_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.1_1 2019-11-29T12:15:03.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__multipart_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.2 2019-11-29T12:15:03.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__shadow_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.2_1 2019-11-29T12:15:03.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__multipart_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.3 2019-11-29T12:15:03.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__shadow_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.3_1 2019-11-29T12:15:03.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__multipart_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.4 2019-11-29T12:15:03.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__shadow_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.4_1 2019-11-29T12:15:03.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:03.839+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.119__multipart_key1/data/file.mpg.2~wPzo_d4cdsmhzIDak4dpxaCVUjiHJ_d.5 2019-11-29T12:15:04.352 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.350+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:04.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.458+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T12:15:04.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.503+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:04.524 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.522+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process iterating over entry tag='42bb32be-1e05-491f-815b-a606bdf24d77.4465.1186', time=2019-11-29T13:59:04.655352+0000, chain.objs.size()=9 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.1 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.1_1 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.2 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.2_1 2019-11-29T12:15:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.3 2019-11-29T12:15:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.3_1 2019-11-29T12:15:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.4 2019-11-29T12:15:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__shadow_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.4_1 2019-11-29T12:15:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.536+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:42bb32be-1e05-491f-815b-a606bdf24d77.4355.73__multipart_file.mpg.2~zb5yXBf8HCPzMbz-45j6KDbKo7t6-S_.5 2019-11-29T12:15:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:04.960+0000 7f8ec90e9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-29T12:15:05.223 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.215+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T12:15:05.254 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.252+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:05.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.267+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:05.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.275+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:05.309 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.307+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T12:15:05.354 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.351+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.369+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.383+0000 7f8ec90e9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:05.448 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.446+0000 7f8ec90e9ec0 20 remove_watcher() i=0 2019-11-29T12:15:05.448 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.446+0000 7f8ec90e9ec0 2 removed watcher, disabling cache 2019-11-29T12:15:05.498 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.496+0000 7f8ec90e9ec0 20 remove_watcher() i=1 2019-11-29T12:15:05.525 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.523+0000 7f8ec90e9ec0 20 remove_watcher() i=2 2019-11-29T12:15:05.577 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.575+0000 7f8ec90e9ec0 20 remove_watcher() i=3 2019-11-29T12:15:05.616 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.614+0000 7f8ec90e9ec0 20 remove_watcher() i=4 2019-11-29T12:15:05.655 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.652+0000 7f8ec90e9ec0 20 remove_watcher() i=5 2019-11-29T12:15:05.690 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.688+0000 7f8ec90e9ec0 20 remove_watcher() i=6 2019-11-29T12:15:05.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.721+0000 7f8ec90e9ec0 20 remove_watcher() i=7 2019-11-29T12:15:05.732 INFO:teuthology.orchestra.run.smithi069:> radosgw-admin gc process --include-all 2019-11-29T12:15:05.860 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.857+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:05.882 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.880+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:05.882 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.880+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:05.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.901+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:05.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.901+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:05.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.909+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:05.911 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.909+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:05.919 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:05.917+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:06.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.099+0000 7f75bc7faec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:06.101 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.099+0000 7f75bc7faec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.099+0000 7f75bc7faec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.099+0000 7f75bc7faec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.099+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.106+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.106+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.116+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.116+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.123+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:06.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.123+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.132+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:06.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.132+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.136+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:06.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.136+0000 7f75bc7faec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:06.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.136+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.141+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:06.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.141+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.149+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.149+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.164+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:06.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.164+0000 7f75bc7faec0 20 zonegroup default 2019-11-29T12:15:06.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.164+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.169+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.169+0000 7f75bc7faec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:06.171 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.169+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.176 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.174+0000 7f75bc7faec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:06.176 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.174+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.182+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:06.184 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.182+0000 7f75bc7faec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:06.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.218+0000 7f75bc7faec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:06.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.218+0000 7f75bc7faec0 20 zone default found 2019-11-29T12:15:06.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.218+0000 7f75bc7faec0 20 started sync module instance, tier type = 2019-11-29T12:15:06.221 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.218+0000 7f75bc7faec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=0 2019-11-29T12:15:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=1 2019-11-29T12:15:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=2 2019-11-29T12:15:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=3 2019-11-29T12:15:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=4 2019-11-29T12:15:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.628+0000 7f75bc7faec0 20 add_watcher() i=5 2019-11-29T12:15:06.651 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.649+0000 7f75bc7faec0 20 add_watcher() i=6 2019-11-29T12:15:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.665+0000 7f75bc7faec0 20 add_watcher() i=7 2019-11-29T12:15:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.665+0000 7f75bc7faec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.665+0000 7f757a7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.670+0000 7f75aa228700 0 -- 172.21.15.69:0/1062651196 >> v2:172.21.15.69:3300/0 conn(0x564c459acf70 msgr2=0x564c459cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T12:15:06.713 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:06.710+0000 7f75a1226700 0 -- 172.21.15.69:0/1623589756 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x564c459fa4c0 msgr2=0x564c459fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:08.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.354+0000 7f74f0ff9700 20 reqs_thread_entry: start 2019-11-29T12:15:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.354+0000 7f74ea7fc700 20 reqs_thread_entry: start 2019-11-29T12:15:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.354+0000 7f75bc7faec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.355+0000 7f74e8ff9700 20 reqs_thread_entry: start 2019-11-29T12:15:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.355+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-29T12:15:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.397+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.412+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.420+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.450+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-29T12:15:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.531+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.558 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.556+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.569+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.615 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.613+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-29T12:15:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.676+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.708 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.700+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:08.714 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.712+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.752+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-29T12:15:08.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.802+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.817+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.826+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.861+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-29T12:15:08.903 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.900+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.921+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.947+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:08.980+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-29T12:15:09.026 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.022+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.039 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.037+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.046+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.078+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-29T12:15:09.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.118+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.136 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.134+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.147 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.145+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.181 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.178+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-29T12:15:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.252+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.308 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.306+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.324 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.323+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.369 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.368+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-29T12:15:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.428+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.452 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.451+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.466 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.465+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.510+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-29T12:15:09.585 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.584+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.605+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.615+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.659+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-29T12:15:09.725 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.724+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.756+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.769 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.768+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.802+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-29T12:15:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.860+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.885 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.883+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:09.893 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.892+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.938+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-29T12:15:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:09.996+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.019 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.018+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.029+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.078+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-29T12:15:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.127+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.144+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.153+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.213+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-29T12:15:10.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.266+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.283 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.282+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.290+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.327+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-29T12:15:10.366 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.364+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.382 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.380+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.390 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.389+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.422 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.420+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-29T12:15:10.462 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.460+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.475+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.484 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.483+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.515+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-29T12:15:10.568 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.567+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.584 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.582+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.592 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.590+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.674+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-29T12:15:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.770+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.791+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.804+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.851+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-29T12:15:10.901 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.900+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.920 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.919+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:10.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.930+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:10.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:10.961+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-29T12:15:11.044 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.043+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.070+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.085 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.083+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.145 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.143+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-29T12:15:11.222 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.221+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.243+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.256 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.254+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.289+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-29T12:15:11.328 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.326+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.343 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.342+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.351 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.350+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.380+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-29T12:15:11.464 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.463+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.504 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.503+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.528+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.614+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-29T12:15:11.700 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.698+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.745 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.744+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.777 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.776+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.853 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.845+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-29T12:15:11.909 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.908+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.924+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:11.936 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.935+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:11.976 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:11.975+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-29T12:15:12.043 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.042+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.089 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.071+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.090 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.085+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.132 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.130+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-29T12:15:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.208+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.233 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.231+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.245 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.244+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.290 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.288+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-29T12:15:12.351 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.348+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.366+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.378 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.376+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.420 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.418+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-29T12:15:12.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.480+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.504+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.522 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.520+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.588 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.586+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-29T12:15:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.644+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.673+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.685 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.684+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.728+0000 7f75bc7faec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-29T12:15:12.800 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.799+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.829 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.828+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-29T12:15:12.859 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.844+0000 7f75bc7faec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-29T12:15:12.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.965+0000 7f75bc7faec0 20 remove_watcher() i=0 2019-11-29T12:15:12.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:12.965+0000 7f75bc7faec0 2 removed watcher, disabling cache 2019-11-29T12:15:13.020 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.019+0000 7f75bc7faec0 20 remove_watcher() i=1 2019-11-29T12:15:13.063 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.061+0000 7f75bc7faec0 20 remove_watcher() i=2 2019-11-29T12:15:13.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.117+0000 7f75bc7faec0 20 remove_watcher() i=3 2019-11-29T12:15:13.181 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.180+0000 7f75bc7faec0 20 remove_watcher() i=4 2019-11-29T12:15:13.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.224+0000 7f75bc7faec0 20 remove_watcher() i=5 2019-11-29T12:15:13.269 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.268+0000 7f75bc7faec0 20 remove_watcher() i=6 2019-11-29T12:15:13.313 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:13.312+0000 7f75bc7faec0 20 remove_watcher() i=7 2019-11-29T12:15:13.322 INFO:teuthology.run_tasks:Running task s3tests... 2019-11-29T12:15:13.334 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-11-29T12:15:13.335 INFO:tasks.s3tests:Downloading s3-tests... 2019-11-29T12:15:13.335 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-11-29T12:15:13.335 INFO:teuthology.orchestra.run.smithi069:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-11-29T12:15:13.428 INFO:teuthology.orchestra.run.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-11-29T12:15:13.882 INFO:tasks.s3tests:Creating rgw users... 2019-11-29T12:15:13.882 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-11-29T12:15:13.883 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key UOMFTAIBECLZKJCOMSGG --secret 0Ril4yKs0UyqOAl0AKHtWGNO4ynaD1zE3cPqLsKuK2GVRsvDVOwD+w== --email bar.client.0+test@test.test --cluster ceph 2019-11-29T12:15:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.024+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.054 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.052+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.055 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.052+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.062+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.062+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.067+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.069 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.067+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.078 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.077+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:14.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.290+0000 7fdb839a3ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:14.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.290+0000 7fdb839a3ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:14.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.290+0000 7fdb839a3ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:14.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.290+0000 7fdb839a3ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:14.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.290+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.302+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:14.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.302+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.330 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.328+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:14.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.328+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.338+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:14.342 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.338+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.354+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:14.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.354+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.362 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.361+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.362 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.361+0000 7fdb839a3ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:14.363 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.361+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.370 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.368+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.370 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.368+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.376+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:14.377 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.376+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.394+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.394+0000 7fdb839a3ec0 20 zonegroup default 2019-11-29T12:15:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.394+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.400+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.400+0000 7fdb839a3ec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.400+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.406+0000 7fdb839a3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.406+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.413+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.413+0000 7fdb839a3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:14.429 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.428+0000 7fdb839a3ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.428+0000 7fdb839a3ec0 20 zone default found 2019-11-29T12:15:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.428+0000 7fdb839a3ec0 20 started sync module instance, tier type = 2019-11-29T12:15:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.428+0000 7fdb839a3ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:14.779 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=0 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=1 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=2 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=3 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=4 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=5 2019-11-29T12:15:14.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.777+0000 7fdb839a3ec0 20 add_watcher() i=6 2019-11-29T12:15:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.812+0000 7fdb839a3ec0 20 add_watcher() i=7 2019-11-29T12:15:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.812+0000 7fdb839a3ec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.812+0000 7fdb417e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:14.829 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.828+0000 7fdb703cf700 0 -- >> v2:172.21.15.69:3300/0 conn(0x559bc1414c50 msgr2=0x559bc1417160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:14.870 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:14.869+0000 7fdb70bd0700 0 -- 172.21.15.69:0/4242164132 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7fdab401a320 msgr2=0x7fdab401c7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:16.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.434+0000 7fdaab7fe700 20 reqs_thread_entry: start 2019-11-29T12:15:16.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.434+0000 7fdaa97fa700 20 reqs_thread_entry: start 2019-11-29T12:15:16.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.434+0000 7fdb839a3ec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:16.436 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.435+0000 7fda9b7fe700 20 reqs_thread_entry: start 2019-11-29T12:15:16.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.435+0000 7fdb839a3ec0 20 get_system_obj_state: rctx=0x7ffd03535df8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x559bc14820d0 s->prefetch_data=0 2019-11-29T12:15:16.437 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.435+0000 7fdb839a3ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-29T12:15:16.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.445+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-11-29T12:15:16.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.445+0000 7fdb839a3ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:16.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.445+0000 7fdb839a3ec0 20 get_system_obj_state: rctx=0x7ffd03535e08 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x559bc1483e60 s->prefetch_data=0 2019-11-29T12:15:16.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.445+0000 7fdb839a3ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-11-29T12:15:16.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.454+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-11-29T12:15:16.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.454+0000 7fdb839a3ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T12:15:16.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.454+0000 7fdb839a3ec0 20 get_system_obj_state: rctx=0x7ffd03535e08 obj=default.rgw.meta:users.keys:UOMFTAIBECLZKJCOMSGG state=0x559bc1483e60 s->prefetch_data=0 2019-11-29T12:15:16.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.454+0000 7fdb839a3ec0 10 cache get: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG : miss 2019-11-29T12:15:16.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.462+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG info.flags=0x0 2019-11-29T12:15:16.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.462+0000 7fdb839a3ec0 10 adding default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG to cache LRU end 2019-11-29T12:15:16.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.462+0000 7fdb839a3ec0 20 get_system_obj_state: rctx=0x7ffd03535a18 obj=default.rgw.meta:users.keys:UOMFTAIBECLZKJCOMSGG state=0x559bc1483e60 s->prefetch_data=0 2019-11-29T12:15:16.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.462+0000 7fdb839a3ec0 10 cache get: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:16.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.469+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG info.flags=0x0 2019-11-29T12:15:16.470 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.469+0000 7fdb839a3ec0 10 moving default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG to cache LRU end 2019-11-29T12:15:16.471 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.469+0000 7fdb839a3ec0 20 get_system_obj_state: rctx=0x7ffd03535b98 obj=default.rgw.meta:users.keys:UOMFTAIBECLZKJCOMSGG state=0x559bc1483e60 s->prefetch_data=0 2019-11-29T12:15:16.471 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.469+0000 7fdb839a3ec0 10 cache get: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.476+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG info.flags=0x0 2019-11-29T12:15:16.479 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.476+0000 7fdb839a3ec0 10 moving default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG to cache LRU end 2019-11-29T12:15:16.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.517+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T12:15:16.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.517+0000 7fdb839a3ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:16.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.517+0000 7fdb839a3ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-11-29T12:15:16.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.527+0000 7fdb567fc700 10 RGWWatcher::handle_notify() notify_id 571230650393 cookie 94127450424672 notifier 4950 bl.length()=617 2019-11-29T12:15:16.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.527+0000 7fdb567fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T12:15:16.536 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:16.527+0000 7fdb567fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.354+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.354+0000 7fdb839a3ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T12:15:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.354+0000 7fdb839a3ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-11-29T12:15:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.366+0000 7fdb567fc700 10 RGWWatcher::handle_notify() notify_id 571230650394 cookie 94127450424672 notifier 4950 bl.length()=194 2019-11-29T12:15:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.366+0000 7fdb567fc700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.366+0000 7fdb567fc700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T12:15:17.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.963+0000 7fdb839a3ec0 10 cache put: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG info.flags=0x7 2019-11-29T12:15:17.964 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.963+0000 7fdb839a3ec0 10 moving default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG to cache LRU end 2019-11-29T12:15:17.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.963+0000 7fdb839a3ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=186 2019-11-29T12:15:17.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.974+0000 7fdb567fc700 10 RGWWatcher::handle_notify() notify_id 571230650431 cookie 94127450448432 notifier 4950 bl.length()=186 2019-11-29T12:15:17.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.974+0000 7fdb567fc700 10 cache put: name=default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG info.flags=0x7 2019-11-29T12:15:17.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:17.974+0000 7fdb567fc700 10 moving default.rgw.meta+users.keys+UOMFTAIBECLZKJCOMSGG to cache LRU end 2019-11-29T12:15:18.208 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T12:15:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "bar.client.0", 2019-11-29T12:15:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. bar.client.0", 2019-11-29T12:15:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "bar.client.0+test@test.test", 2019-11-29T12:15:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T12:15:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "bar.client.0", 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "UOMFTAIBECLZKJCOMSGG", 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "0Ril4yKs0UyqOAl0AKHtWGNO4ynaD1zE3cPqLsKuK2GVRsvDVOwD+w==" 2019-11-29T12:15:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T12:15:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T12:15:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T12:15:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T12:15:18.214 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T12:15:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:18.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.284+0000 7fdb839a3ec0 20 remove_watcher() i=0 2019-11-29T12:15:18.293 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.284+0000 7fdb839a3ec0 2 removed watcher, disabling cache 2019-11-29T12:15:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.322+0000 7fdb839a3ec0 20 remove_watcher() i=1 2019-11-29T12:15:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.356+0000 7fdb839a3ec0 20 remove_watcher() i=2 2019-11-29T12:15:18.424 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.423+0000 7fdb839a3ec0 20 remove_watcher() i=3 2019-11-29T12:15:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.466+0000 7fdb839a3ec0 20 remove_watcher() i=4 2019-11-29T12:15:18.517 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.516+0000 7fdb839a3ec0 20 remove_watcher() i=5 2019-11-29T12:15:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.581+0000 7fdb839a3ec0 20 remove_watcher() i=6 2019-11-29T12:15:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.648+0000 7fdb839a3ec0 20 remove_watcher() i=7 2019-11-29T12:15:18.657 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 2811976310 --totp-seed D7INTFGO26UAXAD7I47Y3XO3JH7TYPRSYVX24AHSF2EY3QJQQ2ZA3OZGR5UBLGYD --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-29T12:15:18.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.803+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:18.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.823+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:18.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.823+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:18.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.832+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:18.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.832+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.838+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.838+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:18.848+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:19.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.063+0000 7fd731be3ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:19.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.063+0000 7fd731be3ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:19.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.063+0000 7fd731be3ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:19.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.063+0000 7fd731be3ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:19.065 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.063+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.078+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:19.080 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.078+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.099 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.098+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:19.100 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.098+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.108+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:19.110 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.108+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.126+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:19.128 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.126+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.137 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.134+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:19.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.134+0000 7fd731be3ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:19.138 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.134+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.141+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:19.144 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.141+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.150+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:19.151 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.150+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.167+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.167+0000 7fd731be3ec0 20 zonegroup default 2019-11-29T12:15:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.167+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.177+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:19.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.177+0000 7fd731be3ec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:19.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.177+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.186+0000 7fd731be3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:19.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.186+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.196+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.196+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:19.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.209+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:19.211 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.209+0000 7fd731be3ec0 20 zone default found 2019-11-29T12:15:19.211 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.209+0000 7fd731be3ec0 20 started sync module instance, tier type = 2019-11-29T12:15:19.211 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.209+0000 7fd731be3ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:19.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=0 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=1 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=2 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=3 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=4 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=5 2019-11-29T12:15:19.647 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.645+0000 7fd731be3ec0 20 add_watcher() i=6 2019-11-29T12:15:19.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.675+0000 7fd731be3ec0 20 add_watcher() i=7 2019-11-29T12:15:19.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.675+0000 7fd731be3ec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:19.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.676+0000 7fd6ef7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.684+0000 7fd71e60f700 0 -- 172.21.15.69:0/1486360892 >> v2:172.21.15.163:3300/0 conn(0x560e6bc005d0 msgr2=0x560e6bc02a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T12:15:19.722 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:19.712+0000 7fd71e60f700 0 -- 172.21.15.69:0/2023255300 >> v1:172.21.15.69:6789/0 conn(0x560e6bc06640 legacy=0x560e6bc08b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:21.153 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.151+0000 7fd65dffb700 20 reqs_thread_entry: start 2019-11-29T12:15:21.153 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.152+0000 7fd6577fe700 20 reqs_thread_entry: start 2019-11-29T12:15:21.153 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.152+0000 7fd731be3ec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:21.154 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.152+0000 7fd655ffb700 20 reqs_thread_entry: start 2019-11-29T12:15:21.154 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.152+0000 7fd731be3ec0 20 get_system_obj_state: rctx=0x7fff4dbf9c28 obj=default.rgw.meta:users.uid:bar.client.0 state=0x560e6bc7f450 s->prefetch_data=0 2019-11-29T12:15:21.154 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.152+0000 7fd731be3ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-29T12:15:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-11-29T12:15:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T12:15:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T12:15:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T12:15:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.185+0000 7fd731be3ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:21.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.209+0000 7fd731be3ec0 20 rados_obj.operate() r=0 bl.length=432 2019-11-29T12:15:21.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.209+0000 7fd731be3ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-11-29T12:15:21.210 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:21.209+0000 7fd731be3ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:23.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.190+0000 7fd731be3ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T12:15:23.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.190+0000 7fd731be3ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:23.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.190+0000 7fd731be3ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-11-29T12:15:23.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.208+0000 7fd6fffff700 10 RGWWatcher::handle_notify() notify_id 579820584987 cookie 94619937146544 notifier 4968 bl.length()=631 2019-11-29T12:15:23.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.208+0000 7fd6fffff700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T12:15:23.209 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.208+0000 7fd6fffff700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T12:15:23.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.392+0000 7fd731be3ec0 20 remove_watcher() i=0 2019-11-29T12:15:23.393 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.392+0000 7fd731be3ec0 2 removed watcher, disabling cache 2019-11-29T12:15:23.431 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.430+0000 7fd731be3ec0 20 remove_watcher() i=1 2019-11-29T12:15:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.462+0000 7fd731be3ec0 20 remove_watcher() i=2 2019-11-29T12:15:23.520 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.518+0000 7fd731be3ec0 20 remove_watcher() i=3 2019-11-29T12:15:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.594+0000 7fd731be3ec0 20 remove_watcher() i=4 2019-11-29T12:15:23.630 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.629+0000 7fd731be3ec0 20 remove_watcher() i=5 2019-11-29T12:15:23.975 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:23.974+0000 7fd731be3ec0 20 remove_watcher() i=6 2019-11-29T12:15:24.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.189+0000 7fd731be3ec0 20 remove_watcher() i=7 2019-11-29T12:15:24.197 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-11-29T12:15:24.197 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key HHBACIXZJFYTJJFSRMUE --secret c6AQ00GoNz37UWmulmRCI8sgS8fhR1Kokag2DKVrggcY3j9xjs5Iwg== --email foo.client.0+test@test.test --cluster ceph 2019-11-29T12:15:24.336 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.335+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.356+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.357 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.356+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.365 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.364+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.365 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.364+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.370+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.370+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.381 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.380+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:24.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.509+0000 7f95e987bec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:24.510 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.509+0000 7f95e987bec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:24.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.509+0000 7f95e987bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:24.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.509+0000 7f95e987bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:24.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.509+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.517+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.517+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.530+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.530+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.543+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:24.544 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.543+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.558+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:24.560 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.558+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.573+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.573+0000 7f95e987bec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.573+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.578+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.578+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.587+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:24.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.587+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.595+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.595+0000 7f95e987bec0 20 zonegroup default 2019-11-29T12:15:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.595+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.599+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.599+0000 7f95e987bec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:24.601 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.599+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.604+0000 7f95e987bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.604+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.612+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.612+0000 7f95e987bec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.624+0000 7f95e987bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.624+0000 7f95e987bec0 20 zone default found 2019-11-29T12:15:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.624+0000 7f95e987bec0 20 started sync module instance, tier type = 2019-11-29T12:15:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.624+0000 7f95e987bec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:24.931 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=0 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=1 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=2 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=3 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=4 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=5 2019-11-29T12:15:24.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.930+0000 7f95e987bec0 20 add_watcher() i=6 2019-11-29T12:15:24.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.960+0000 7f95e987bec0 20 add_watcher() i=7 2019-11-29T12:15:24.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.960+0000 7f95e987bec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:24.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.960+0000 7f95a77de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:24.976 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:24.975+0000 7f95d72a9700 0 -- 172.21.15.69:0/2036112087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623d6e9fd10 msgr2=0x5623d6ea2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:25.053+0000 7f95d72a9700 0 -- 172.21.15.69:0/2028001990 >> [v2:172.21.15.69:6818/9728,v1:172.21.15.69:6819/9728] conn(0x5623d6ebcf60 msgr2=0x5623d6ebf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:26.727 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.726+0000 7f9515ffb700 20 reqs_thread_entry: start 2019-11-29T12:15:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.726+0000 7f950f7fe700 20 reqs_thread_entry: start 2019-11-29T12:15:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.726+0000 7f95e987bec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.727+0000 7f950dffb700 20 reqs_thread_entry: start 2019-11-29T12:15:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.727+0000 7f95e987bec0 20 get_system_obj_state: rctx=0x7fff22293c78 obj=default.rgw.meta:users.uid:foo.client.0 state=0x5623d6f26d40 s->prefetch_data=0 2019-11-29T12:15:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.727+0000 7f95e987bec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-29T12:15:26.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.744+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-11-29T12:15:26.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.744+0000 7f95e987bec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:26.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.744+0000 7f95e987bec0 20 get_system_obj_state: rctx=0x7fff22293c88 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x5623d6f28ad0 s->prefetch_data=0 2019-11-29T12:15:26.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.744+0000 7f95e987bec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-11-29T12:15:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.751+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-11-29T12:15:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.751+0000 7f95e987bec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T12:15:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.751+0000 7f95e987bec0 20 get_system_obj_state: rctx=0x7fff22293c88 obj=default.rgw.meta:users.keys:HHBACIXZJFYTJJFSRMUE state=0x5623d6f28ad0 s->prefetch_data=0 2019-11-29T12:15:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.751+0000 7f95e987bec0 10 cache get: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE : miss 2019-11-29T12:15:26.781 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.780+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE info.flags=0x0 2019-11-29T12:15:26.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.780+0000 7f95e987bec0 10 adding default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE to cache LRU end 2019-11-29T12:15:26.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.780+0000 7f95e987bec0 20 get_system_obj_state: rctx=0x7fff22293898 obj=default.rgw.meta:users.keys:HHBACIXZJFYTJJFSRMUE state=0x5623d6f28ad0 s->prefetch_data=0 2019-11-29T12:15:26.782 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.780+0000 7f95e987bec0 10 cache get: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:26.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.789+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE info.flags=0x0 2019-11-29T12:15:26.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.789+0000 7f95e987bec0 10 moving default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE to cache LRU end 2019-11-29T12:15:26.790 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.789+0000 7f95e987bec0 20 get_system_obj_state: rctx=0x7fff22293a18 obj=default.rgw.meta:users.keys:HHBACIXZJFYTJJFSRMUE state=0x5623d6f28ad0 s->prefetch_data=0 2019-11-29T12:15:26.791 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.789+0000 7f95e987bec0 10 cache get: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:26.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.804+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE info.flags=0x0 2019-11-29T12:15:26.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.804+0000 7f95e987bec0 10 moving default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE to cache LRU end 2019-11-29T12:15:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.848+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T12:15:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.848+0000 7f95e987bec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.848+0000 7f95e987bec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-11-29T12:15:26.862 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.860+0000 7f95b7fff700 10 RGWWatcher::handle_notify() notify_id 584115552303 cookie 94711929409184 notifier 4607 bl.length()=617 2019-11-29T12:15:26.862 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.861+0000 7f95b7fff700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T12:15:26.862 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.861+0000 7f95b7fff700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.963+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.963+0000 7f95e987bec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T12:15:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.963+0000 7f95e987bec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-11-29T12:15:26.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.988+0000 7f95b7fff700 10 RGWWatcher::handle_notify() notify_id 584115552324 cookie 94711929416288 notifier 4607 bl.length()=194 2019-11-29T12:15:26.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.988+0000 7f95b7fff700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:26.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:26.988+0000 7f95b7fff700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T12:15:27.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.101+0000 7f95e987bec0 10 cache put: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE info.flags=0x7 2019-11-29T12:15:27.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.101+0000 7f95e987bec0 10 moving default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE to cache LRU end 2019-11-29T12:15:27.102 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.101+0000 7f95e987bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=186 2019-11-29T12:15:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.126+0000 7f95b7fff700 10 RGWWatcher::handle_notify() notify_id 584115552320 cookie 94711929414752 notifier 4607 bl.length()=186 2019-11-29T12:15:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.126+0000 7f95b7fff700 10 cache put: name=default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE info.flags=0x7 2019-11-29T12:15:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.126+0000 7f95b7fff700 10 moving default.rgw.meta+users.keys+HHBACIXZJFYTJJFSRMUE to cache LRU end 2019-11-29T12:15:27.203 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T12:15:27.204 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "foo.client.0", 2019-11-29T12:15:27.204 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. foo.client.0", 2019-11-29T12:15:27.204 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "foo.client.0+test@test.test", 2019-11-29T12:15:27.204 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T12:15:27.204 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "foo.client.0", 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "HHBACIXZJFYTJJFSRMUE", 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "c6AQ00GoNz37UWmulmRCI8sgS8fhR1Kokag2DKVrggcY3j9xjs5Iwg==" 2019-11-29T12:15:27.205 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T12:15:27.206 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:27.207 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T12:15:27.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:27.208 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:27.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:27.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T12:15:27.209 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.251+0000 7f95e987bec0 20 remove_watcher() i=0 2019-11-29T12:15:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.251+0000 7f95e987bec0 2 removed watcher, disabling cache 2019-11-29T12:15:27.298 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.297+0000 7f95e987bec0 20 remove_watcher() i=1 2019-11-29T12:15:27.350 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.349+0000 7f95e987bec0 20 remove_watcher() i=2 2019-11-29T12:15:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.418+0000 7f95e987bec0 20 remove_watcher() i=3 2019-11-29T12:15:27.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.466+0000 7f95e987bec0 20 remove_watcher() i=4 2019-11-29T12:15:27.514 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.513+0000 7f95e987bec0 20 remove_watcher() i=5 2019-11-29T12:15:27.566 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.565+0000 7f95e987bec0 20 remove_watcher() i=6 2019-11-29T12:15:27.622 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.621+0000 7f95e987bec0 20 remove_watcher() i=7 2019-11-29T12:15:27.633 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 3311618910 --totp-seed RZBXQPSWAIMK3I4JXUPNOK4YW7OPRQ5VFN5GKQCHJM64OGFIGK43K4SMBG2PBSZV --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-29T12:15:27.780 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.779+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.804+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.804+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:27.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.813+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:27.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.813+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:27.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.818+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:27.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.818+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:27.830 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:27.829+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:28.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.074+0000 7fdbfd29dec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:28.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.074+0000 7fdbfd29dec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:28.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.074+0000 7fdbfd29dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:28.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.074+0000 7fdbfd29dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:28.076 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.074+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.086 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.085+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:28.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.085+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.106 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.105+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:28.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.105+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.116+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:28.118 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.116+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.133+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:28.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.133+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.141+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:28.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.141+0000 7fdbfd29dec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:28.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.141+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.149 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.146+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:28.149 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.146+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.154+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.155+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.173+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.173+0000 7fdbfd29dec0 20 zonegroup default 2019-11-29T12:15:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.173+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.184+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:28.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.184+0000 7fdbfd29dec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:28.186 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.184+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.191 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.190+0000 7fdbfd29dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:28.300 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.190+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.197+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:28.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.197+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:28.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.212+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:28.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.212+0000 7fdbfd29dec0 20 zone default found 2019-11-29T12:15:28.301 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.212+0000 7fdbfd29dec0 20 started sync module instance, tier type = 2019-11-29T12:15:28.302 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.212+0000 7fdbfd29dec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=0 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=1 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=2 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=3 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=4 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=5 2019-11-29T12:15:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.669+0000 7fdbfd29dec0 20 add_watcher() i=6 2019-11-29T12:15:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.705+0000 7fdbfd29dec0 20 add_watcher() i=7 2019-11-29T12:15:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.705+0000 7fdbfd29dec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.706+0000 7fdbbafdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.718+0000 7fdbeaccb700 0 -- >> v2:172.21.15.69:3300/0 conn(0x55936ef91970 msgr2=0x55936ef91d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T12:15:28.784 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:28.783+0000 7fdbeaccb700 0 -- 172.21.15.69:0/3627726425 >> [v2:172.21.15.69:6818/9728,v1:172.21.15.69:6819/9728] conn(0x55936efa2a00 msgr2=0x55936efa2e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:30.453 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.452+0000 7fdb297fa700 20 reqs_thread_entry: start 2019-11-29T12:15:30.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.452+0000 7fdb22ffd700 20 reqs_thread_entry: start 2019-11-29T12:15:30.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.452+0000 7fdbfd29dec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:30.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.453+0000 7fdb217fa700 20 reqs_thread_entry: start 2019-11-29T12:15:30.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.453+0000 7fdbfd29dec0 20 get_system_obj_state: rctx=0x7ffc5e3b4758 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55936f010450 s->prefetch_data=0 2019-11-29T12:15:30.455 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.453+0000 7fdbfd29dec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-29T12:15:30.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-11-29T12:15:30.487 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:30.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T12:15:30.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T12:15:30.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T12:15:30.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.486+0000 7fdbfd29dec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.500+0000 7fdbfd29dec0 20 rados_obj.operate() r=0 bl.length=432 2019-11-29T12:15:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.500+0000 7fdbfd29dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-11-29T12:15:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.500+0000 7fdbfd29dec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:30.716 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.710+0000 7fdbfd29dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T12:15:30.716 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.710+0000 7fdbfd29dec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:30.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.710+0000 7fdbfd29dec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-11-29T12:15:30.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.735+0000 7fdbcb7fe700 10 RGWWatcher::handle_notify() notify_id 584115552304 cookie 94091710253120 notifier 4616 bl.length()=631 2019-11-29T12:15:30.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.735+0000 7fdbcb7fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T12:15:30.737 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.735+0000 7fdbcb7fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T12:15:30.945 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.939+0000 7fdbfd29dec0 20 remove_watcher() i=0 2019-11-29T12:15:30.945 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.939+0000 7fdbfd29dec0 2 removed watcher, disabling cache 2019-11-29T12:15:30.983 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:30.982+0000 7fdbfd29dec0 20 remove_watcher() i=1 2019-11-29T12:15:31.030 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.029+0000 7fdbfd29dec0 20 remove_watcher() i=2 2019-11-29T12:15:31.087 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.086+0000 7fdbfd29dec0 20 remove_watcher() i=3 2019-11-29T12:15:31.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.123+0000 7fdbfd29dec0 20 remove_watcher() i=4 2019-11-29T12:15:31.172 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.170+0000 7fdbfd29dec0 20 remove_watcher() i=5 2019-11-29T12:15:31.212 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.210+0000 7fdbfd29dec0 20 remove_watcher() i=6 2019-11-29T12:15:31.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.263+0000 7fdbfd29dec0 20 remove_watcher() i=7 2019-11-29T12:15:31.274 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-11-29T12:15:31.275 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key MQHLCBQGRZVGECKEKFEQ --secret Gb0jR+bEvN56MszHozrp0KRYXkFbH5chjGTtpM3IlCtr2P5ore+HQA== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-11-29T12:15:31.439 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.437+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.459+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.461 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.459+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.468+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.468+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.474 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.473+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.474 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.473+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.482 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.481+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:31.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.703+0000 7f0b42467ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:31.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.704+0000 7f0b42467ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:31.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.704+0000 7f0b42467ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:31.705 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.704+0000 7f0b42467ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:31.706 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.704+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.716 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.715+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:31.717 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.715+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.736 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.734+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:31.736 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.734+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.746+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.746+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.765+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:31.766 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.765+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.771+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.771+0000 7f0b42467ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:31.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.771+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.777+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.778 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.777+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.788 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.786+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:31.788 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.786+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.803+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:31.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.803+0000 7f0b42467ec0 20 zonegroup default 2019-11-29T12:15:31.804 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.803+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.809 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.808+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.810 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.808+0000 7f0b42467ec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:31.810 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.808+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.814+0000 7f0b42467ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:31.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.814+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.822+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:31.841 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.822+0000 7f0b42467ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:31.842 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.841+0000 7f0b42467ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:31.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.841+0000 7f0b42467ec0 20 zone default found 2019-11-29T12:15:31.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.841+0000 7f0b42467ec0 20 started sync module instance, tier type = 2019-11-29T12:15:31.843 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:31.841+0000 7f0b42467ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=0 2019-11-29T12:15:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=1 2019-11-29T12:15:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=2 2019-11-29T12:15:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=3 2019-11-29T12:15:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=4 2019-11-29T12:15:32.248 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=5 2019-11-29T12:15:32.248 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.245+0000 7f0b42467ec0 20 add_watcher() i=6 2019-11-29T12:15:32.286 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.284+0000 7f0b42467ec0 20 add_watcher() i=7 2019-11-29T12:15:32.286 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.284+0000 7f0b42467ec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:32.286 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.285+0000 7f0afffdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:32.299 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.298+0000 7f0b27fff700 0 -- >> v2:172.21.15.163:3300/0 conn(0x55d8adefc7a0 msgr2=0x55d8adefec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:32.398 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:32.397+0000 7f0b2f694700 0 -- 172.21.15.69:0/50825729 >> [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] conn(0x55d8adf0d740 msgr2=0x55d8adf0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T12:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.014+0000 7f0a6e7fc700 20 reqs_thread_entry: start 2019-11-29T12:15:34.016 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.014+0000 7f0a67fff700 20 reqs_thread_entry: start 2019-11-29T12:15:34.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.014+0000 7f0b42467ec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:34.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.015+0000 7f0a667fc700 20 reqs_thread_entry: start 2019-11-29T12:15:34.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.015+0000 7f0b42467ec0 20 get_system_obj_state: rctx=0x7ffdcbf4f998 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55d8adf742c0 s->prefetch_data=0 2019-11-29T12:15:34.018 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.015+0000 7f0b42467ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-29T12:15:34.022 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.021+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-11-29T12:15:34.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.021+0000 7f0b42467ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:34.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.021+0000 7f0b42467ec0 20 get_system_obj_state: rctx=0x7ffdcbf4f9a8 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x55d8adf76070 s->prefetch_data=0 2019-11-29T12:15:34.023 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.021+0000 7f0b42467ec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-11-29T12:15:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.027+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-11-29T12:15:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.027+0000 7f0b42467ec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-29T12:15:34.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.027+0000 7f0b42467ec0 20 get_system_obj_state: rctx=0x7ffdcbf4f9a8 obj=default.rgw.meta:users.keys:MQHLCBQGRZVGECKEKFEQ state=0x55d8adf76070 s->prefetch_data=0 2019-11-29T12:15:34.029 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.027+0000 7f0b42467ec0 10 cache get: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ : miss 2019-11-29T12:15:34.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.039+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ info.flags=0x0 2019-11-29T12:15:34.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.039+0000 7f0b42467ec0 10 adding default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ to cache LRU end 2019-11-29T12:15:34.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.039+0000 7f0b42467ec0 20 get_system_obj_state: rctx=0x7ffdcbf4f5b8 obj=default.rgw.meta:users.keys:MQHLCBQGRZVGECKEKFEQ state=0x55d8adf76070 s->prefetch_data=0 2019-11-29T12:15:34.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.039+0000 7f0b42467ec0 10 cache get: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:34.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.048+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ info.flags=0x0 2019-11-29T12:15:34.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.048+0000 7f0b42467ec0 10 moving default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ to cache LRU end 2019-11-29T12:15:34.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.049+0000 7f0b42467ec0 20 get_system_obj_state: rctx=0x7ffdcbf4f738 obj=default.rgw.meta:users.keys:MQHLCBQGRZVGECKEKFEQ state=0x55d8adf76070 s->prefetch_data=0 2019-11-29T12:15:34.050 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.049+0000 7f0b42467ec0 10 cache get: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ : type miss (requested=0x6, cached=0x0) 2019-11-29T12:15:34.062 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.061+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ info.flags=0x0 2019-11-29T12:15:34.063 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.061+0000 7f0b42467ec0 10 moving default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ to cache LRU end 2019-11-29T12:15:34.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.116+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-29T12:15:34.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.116+0000 7f0b42467ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:34.125 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.116+0000 7f0b42467ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-11-29T12:15:34.126 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.125+0000 7f0b24ff9700 10 RGWWatcher::handle_notify() notify_id 584115552321 cookie 94389119354208 notifier 4628 bl.length()=691 2019-11-29T12:15:34.126 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.125+0000 7f0b24ff9700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-29T12:15:34.126 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.125+0000 7f0b24ff9700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:34.498 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.497+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:34.498 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.497+0000 7f0b42467ec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-29T12:15:34.499 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.497+0000 7f0b42467ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-11-29T12:15:34.519 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.516+0000 7f0b24ff9700 10 RGWWatcher::handle_notify() notify_id 584115552325 cookie 94389119368512 notifier 4628 bl.length()=224 2019-11-29T12:15:34.519 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.516+0000 7f0b24ff9700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-11-29T12:15:34.519 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.516+0000 7f0b24ff9700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-29T12:15:34.695 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.694+0000 7f0b42467ec0 10 cache put: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ info.flags=0x7 2019-11-29T12:15:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.694+0000 7f0b42467ec0 10 moving default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ to cache LRU end 2019-11-29T12:15:34.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.694+0000 7f0b42467ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=201 2019-11-29T12:15:34.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.708+0000 7f0b24ff9700 10 RGWWatcher::handle_notify() notify_id 584115552284 cookie 94389119328400 notifier 4628 bl.length()=201 2019-11-29T12:15:34.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.708+0000 7f0b24ff9700 10 cache put: name=default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ info.flags=0x7 2019-11-29T12:15:34.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:34.708+0000 7f0b24ff9700 10 moving default.rgw.meta+users.keys+MQHLCBQGRZVGECKEKFEQ to cache LRU end 2019-11-29T12:15:35.342 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T12:15:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "testx$tenanteduser.client.0", 2019-11-29T12:15:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-11-29T12:15:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-11-29T12:15:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [], 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [ 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "testx$tenanteduser.client.0", 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "access_key": "MQHLCBQGRZVGECKEKFEQ", 2019-11-29T12:15:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "Gb0jR+bEvN56MszHozrp0KRYXkFbH5chjGTtpM3IlCtr2P5ore+HQA==" 2019-11-29T12:15:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T12:15:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T12:15:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [], 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T12:15:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T12:15:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T12:15:35.349 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T12:15:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.393+0000 7f0b42467ec0 20 remove_watcher() i=0 2019-11-29T12:15:35.394 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.393+0000 7f0b42467ec0 2 removed watcher, disabling cache 2019-11-29T12:15:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.445+0000 7f0b42467ec0 20 remove_watcher() i=1 2019-11-29T12:15:35.494 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.492+0000 7f0b42467ec0 20 remove_watcher() i=2 2019-11-29T12:15:35.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.553+0000 7f0b42467ec0 20 remove_watcher() i=3 2019-11-29T12:15:35.614 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.613+0000 7f0b42467ec0 20 remove_watcher() i=4 2019-11-29T12:15:35.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.666+0000 7f0b42467ec0 20 remove_watcher() i=5 2019-11-29T12:15:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.711+0000 7f0b42467ec0 20 remove_watcher() i=6 2019-11-29T12:15:35.758 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.757+0000 7f0b42467ec0 20 remove_watcher() i=7 2019-11-29T12:15:35.771 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 5866213730 --totp-seed R42E4YJDN4E3XS7ZWX22PIZC4XSU2QLGSB62P5Q6ULDX3B3KL2M3EDMYVEQPZHJH --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-29T12:15:35.928 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.926+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:35.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.949+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:35.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.949+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:35.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.959+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:35.961 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.959+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:35.965 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.963+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:35.966 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.963+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:35.972 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:35.970+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:36.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.164+0000 7f0f9f459ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T12:15:36.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.164+0000 7f0f9f459ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T12:15:36.166 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.164+0000 7f0f9f459ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T12:15:36.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.164+0000 7f0f9f459ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T12:15:36.167 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.164+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.179 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.177+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:36.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.177+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.203+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.203+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.215+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:36.218 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.215+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.235+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:36.237 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.235+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.241+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:36.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.241+0000 7f0f9f459ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T12:15:36.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.241+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.247+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:36.249 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.247+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.260+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.260+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.281+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T12:15:36.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.281+0000 7f0f9f459ec0 20 zonegroup default 2019-11-29T12:15:36.284 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.281+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.290+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:36.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.290+0000 7f0f9f459ec0 10 Cannot find current period zone using local zone 2019-11-29T12:15:36.292 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.290+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.297+0000 7f0f9f459ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T12:15:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.297+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.311 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.308+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T12:15:36.311 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.308+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:36.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.324+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T12:15:36.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.324+0000 7f0f9f459ec0 20 zone default found 2019-11-29T12:15:36.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.324+0000 7f0f9f459ec0 20 started sync module instance, tier type = 2019-11-29T12:15:36.332 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.324+0000 7f0f9f459ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T12:15:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=0 2019-11-29T12:15:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=1 2019-11-29T12:15:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=2 2019-11-29T12:15:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=3 2019-11-29T12:15:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=4 2019-11-29T12:15:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=5 2019-11-29T12:15:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.675+0000 7f0f9f459ec0 20 add_watcher() i=6 2019-11-29T12:15:36.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.702+0000 7f0f9f459ec0 20 add_watcher() i=7 2019-11-29T12:15:36.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.702+0000 7f0f9f459ec0 2 all 8 watchers are set, enabling cache 2019-11-29T12:15:36.704 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.702+0000 7f0f5cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T12:15:36.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.707+0000 7f0f8ce87700 0 -- 172.21.15.69:0/713621678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558240c156f0 msgr2=0x558240c15b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-29T12:15:36.912 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.909+0000 7f0f8ce87700 0 --2- 172.21.15.69:0/713621678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558240c156f0 0x558240c15b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-29T12:15:36.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:36.969+0000 7f0f83e85700 0 -- 172.21.15.69:0/852375620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558240c22cf0 msgr2=0x558240c25200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T12:15:38.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.562+0000 7f0eceffd700 20 reqs_thread_entry: start 2019-11-29T12:15:38.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.562+0000 7f0eccff9700 20 reqs_thread_entry: start 2019-11-29T12:15:38.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.562+0000 7f0f9f459ec0 20 init_complete bucket index max shards: 0 2019-11-29T12:15:38.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.562+0000 7f0ec2ffd700 20 reqs_thread_entry: start 2019-11-29T12:15:38.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.562+0000 7f0f9f459ec0 20 get_system_obj_state: rctx=0x7ffc4842ee58 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x558240c97bc0 s->prefetch_data=0 2019-11-29T12:15:38.566 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.563+0000 7f0f9f459ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-29T12:15:38.578 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-11-29T12:15:38.578 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:38.578 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T12:15:38.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T12:15:38.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T12:15:38.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.576+0000 7f0f9f459ec0 20 rados->read ofs=0 len=0 2019-11-29T12:15:38.589 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.587+0000 7f0f9f459ec0 20 rados_obj.operate() r=0 bl.length=491 2019-11-29T12:15:38.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.587+0000 7f0f9f459ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-11-29T12:15:38.590 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.587+0000 7f0f9f459ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:38.763 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.761+0000 7f0f9f459ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-29T12:15:38.763 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.761+0000 7f0f9f459ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:38.764 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.761+0000 7f0f9f459ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-11-29T12:15:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.770+0000 7f0f81e81700 10 RGWWatcher::handle_notify() notify_id 584115552322 cookie 94017920450240 notifier 4640 bl.length()=705 2019-11-29T12:15:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.770+0000 7f0f81e81700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-29T12:15:38.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.770+0000 7f0f81e81700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T12:15:39.001 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.999+0000 7f0f9f459ec0 20 remove_watcher() i=0 2019-11-29T12:15:39.001 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:38.999+0000 7f0f9f459ec0 2 removed watcher, disabling cache 2019-11-29T12:15:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.034+0000 7f0f9f459ec0 20 remove_watcher() i=1 2019-11-29T12:15:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.070+0000 7f0f9f459ec0 20 remove_watcher() i=2 2019-11-29T12:15:39.146 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.144+0000 7f0f9f459ec0 20 remove_watcher() i=3 2019-11-29T12:15:39.180 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.178+0000 7f0f9f459ec0 20 remove_watcher() i=4 2019-11-29T12:15:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.229+0000 7f0f9f459ec0 20 remove_watcher() i=5 2019-11-29T12:15:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.286+0000 7f0f9f459ec0 20 remove_watcher() i=6 2019-11-29T12:15:39.338 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T12:15:39.336+0000 7f0f9f459ec0 20 remove_watcher() i=7 2019-11-29T12:15:39.346 INFO:tasks.s3tests:Configuring s3-tests... 2019-11-29T12:15:39.347 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-11-29T12:15:44.625 INFO:teuthology.orchestra.run.smithi069.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-11-29T12:15:44.737 INFO:teuthology.orchestra.run.smithi069.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-29T12:15:44.775 INFO:teuthology.orchestra.run.smithi069.stdout:Loading mirror speeds from cached hostfile 2019-11-29T12:15:44.776 INFO:teuthology.orchestra.run.smithi069.stdout: * base: mirror.facebook.net 2019-11-29T12:15:44.776 INFO:teuthology.orchestra.run.smithi069.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-29T12:15:44.776 INFO:teuthology.orchestra.run.smithi069.stdout: * extras: mirrors.greenmountainaccess.net 2019-11-29T12:15:44.777 INFO:teuthology.orchestra.run.smithi069.stdout: * updates: mirror.facebook.net 2019-11-29T12:15:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:7 packages excluded due to repository priority protections 2019-11-29T12:15:45.831 INFO:teuthology.orchestra.run.smithi069.stdout:Resolving Dependencies 2019-11-29T12:15:45.831 INFO:teuthology.orchestra.run.smithi069.stdout:--> Running transaction check 2019-11-29T12:15:45.832 INFO:teuthology.orchestra.run.smithi069.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-11-29T12:15:46.345 INFO:teuthology.orchestra.run.smithi069.stdout:--> Finished Dependency Resolution 2019-11-29T12:15:46.612 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:46.613 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies Resolved 2019-11-29T12:15:46.613 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-11-29T12:15:46.614 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:46.615 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2019-11-29T12:15:46.615 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2019-11-29T12:15:46.615 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2019-11-29T12:15:46.616 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:46.616 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 23 k 2019-11-29T12:15:46.616 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 27 k 2019-11-29T12:15:46.616 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading packages: 2019-11-29T12:15:46.708 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2019-11-29T12:15:46.711 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2019-11-29T12:15:46.712 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded 2019-11-29T12:15:46.713 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2019-11-29T12:15:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:15:47.645 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2019-11-29T12:15:47.834 INFO:teuthology.orchestra.run.smithi069.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-11-29T12:15:49.204 INFO:teuthology.orchestra.run.smithi069.stdout:Installing setuptools, pip, wheel...done. 2019-11-29T12:15:49.524 INFO:teuthology.orchestra.run.smithi069.stderr:Cache entry deserialization failed, entry ignored 2019-11-29T12:15:49.634 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting pip 2019-11-29T12:15:49.709 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-11-29T12:15:49.771 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: pip 2019-11-29T12:15:49.771 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: pip 9.0.1 2019-11-29T12:15:49.829 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling pip-9.0.1: 2019-11-29T12:15:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled pip-9.0.1 2019-11-29T12:15:50.499 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed pip-19.3.1 2019-11-29T12:15:50.786 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T12:15:51.338 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting setuptools==32.3.1 2019-11-29T12:15:51.342 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-11-29T12:15:51.404 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: setuptools 2019-11-29T12:15:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: setuptools 28.8.0 2019-11-29T12:15:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling setuptools-28.8.0: 2019-11-29T12:15:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-29T12:15:51.630 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed setuptools-32.3.1 2019-11-29T12:15:52.109 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T12:15:52.296 INFO:teuthology.orchestra.run.smithi069.stdout:Processing /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl 2019-11-29T12:15:52.371 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting nose>=1.0.0 2019-11-29T12:15:52.374 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-11-29T12:15:52.542 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting boto>=2.6.0 2019-11-29T12:15:52.549 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-11-29T12:15:53.070 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting boto3>=1.0.0 2019-11-29T12:15:53.118 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/67/17/567f679dac6ec93611ce05637094e9736afa40e41c9ed0aaefd90de6f7e8/boto3-1.10.28-py2.py3-none-any.whl (128kB) 2019-11-29T12:15:53.231 INFO:teuthology.orchestra.run.smithi069.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-11-29T12:15:53.532 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting gevent>=1.0 2019-11-29T12:15:53.553 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T12:15:53.594 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:03:06.485 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.598 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:03:06.489 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.625 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:03:06.515 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.629 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:03:06.519 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.639 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:03:06.413 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.649 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:03:06.423 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.751 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:03:06.641 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.811 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:03:06.700 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:53.851 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting isodate>=0.4.4 2019-11-29T12:15:53.855 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-11-29T12:15:53.882 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:03:06.656 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:15:54.029 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting requests>=0.14.0 2019-11-29T12:15:54.041 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-11-29T12:15:54.304 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting pytz>=2011k 2019-11-29T12:15:54.314 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-11-29T12:15:54.551 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting ordereddict 2019-11-29T12:15:54.561 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-11-29T12:15:54.876 INFO:teuthology.orchestra.run.smithi069.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-11-29T12:15:55.244 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting lxml 2019-11-29T12:15:55.258 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T12:15:55.981 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting botocore<1.14.0,>=1.13.28 2019-11-29T12:15:55.986 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/8e/b3/ab9044d3aa14208f5b6f69665d5f82ad9b028809666e4e70dac1ab0bfd3a/botocore-1.13.28-py2.py3-none-any.whl (5.7MB) 2019-11-29T12:15:56.597 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-11-29T12:15:56.608 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-11-29T12:15:56.724 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-11-29T12:15:56.736 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-11-29T12:15:56.921 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-29T12:15:56.923 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T12:15:56.997 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting six 2019-11-29T12:15:57.008 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-11-29T12:15:57.083 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-11-29T12:15:57.092 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-11-29T12:15:57.173 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting idna<2.9,>=2.5 2019-11-29T12:15:57.180 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-11-29T12:15:57.319 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-11-29T12:15:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-11-29T12:15:57.468 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting certifi>=2017.4.17 2019-11-29T12:15:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-11-29T12:15:57.569 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting docutils<0.16,>=0.10 2019-11-29T12:15:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-11-29T12:15:57.733 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-11-29T12:15:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-11-29T12:15:57.849 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-11-29T12:15:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-11-29T12:15:57.894 INFO:teuthology.orchestra.run.smithi069.stdout:Building wheels for collected packages: ordereddict 2019-11-29T12:15:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for ordereddict (setup.py): started 2019-11-29T12:15:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-11-29T12:15:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=67376160e23a3d2916060a8e7536db4edb82bcb20fdbff4e21f4db949cfd8294 2019-11-29T12:15:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-11-29T12:15:58.135 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully built ordereddict 2019-11-29T12:15:58.221 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-11-29T12:16:00.457 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed PyYAML-5.1.2 boto-2.49.0 boto3-1.10.28 botocore-1.13.28 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-11-29T12:16:00.776 INFO:teuthology.orchestra.run.smithi069.stdout:running develop 2019-11-29T12:16:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:running egg_info 2019-11-29T12:16:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:creating s3tests.egg-info 2019-11-29T12:16:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-11-29T12:16:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:writing s3tests.egg-info/PKG-INFO 2019-11-29T12:16:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-11-29T12:16:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-11-29T12:16:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-11-29T12:16:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-29T12:16:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-29T12:16:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-29T12:16:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:running build_ext 2019-11-29T12:16:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-11-29T12:16:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-11-29T12:16:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-29T12:16:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-29T12:16:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-29T12:16:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-29T12:16:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-29T12:16:00.877 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-11-29T12:16:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:Processing dependencies for s3tests==0.0.1 2019-11-29T12:16:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for isodate==0.6.0 2019-11-29T12:16:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: isodate 0.6.0 2019-11-29T12:16:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-11-29T12:16:00.959 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.960 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.960 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for gevent==1.4.0 2019-11-29T12:16:00.960 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: gevent 1.4.0 2019-11-29T12:16:00.961 INFO:teuthology.orchestra.run.smithi069.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-11-29T12:16:00.961 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.961 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.962 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for bunch==1.0.1 2019-11-29T12:16:00.962 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: bunch 1.0.1 2019-11-29T12:16:00.962 INFO:teuthology.orchestra.run.smithi069.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-11-29T12:16:00.962 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.962 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.963 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for PyYAML==5.1.2 2019-11-29T12:16:00.963 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: PyYAML 5.1.2 2019-11-29T12:16:00.963 INFO:teuthology.orchestra.run.smithi069.stdout:Adding PyYAML 5.1.2 to easy-install.pth file 2019-11-29T12:16:00.963 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.963 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.964 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for boto3==1.10.28 2019-11-29T12:16:00.964 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: boto3 1.10.28 2019-11-29T12:16:00.970 INFO:teuthology.orchestra.run.smithi069.stdout:Adding boto3 1.10.28 to easy-install.pth file 2019-11-29T12:16:00.971 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.971 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for boto==2.49.0 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: boto 2.49.0 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for six==1.13.0 2019-11-29T12:16:00.974 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: six 1.13.0 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout:Adding six 1.13.0 to easy-install.pth file 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for greenlet==0.4.15 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: greenlet 0.4.15 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-11-29T12:16:00.975 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for s3transfer==0.2.1 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: s3transfer 0.2.1 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.976 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for jmespath==0.9.4 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: jmespath 0.9.4 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for botocore==1.13.28 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: botocore 1.13.28 2019-11-29T12:16:00.977 INFO:teuthology.orchestra.run.smithi069.stdout:Adding botocore 1.13.28 to easy-install.pth file 2019-11-29T12:16:00.978 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.978 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.980 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for futures==3.3.0 2019-11-29T12:16:00.980 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: futures 3.3.0 2019-11-29T12:16:00.980 INFO:teuthology.orchestra.run.smithi069.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-11-29T12:16:00.980 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.981 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.982 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for urllib3==1.25.7 2019-11-29T12:16:00.982 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: urllib3 1.25.7 2019-11-29T12:16:00.982 INFO:teuthology.orchestra.run.smithi069.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-11-29T12:16:00.982 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.982 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.983 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for python-dateutil==2.8.0 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: python-dateutil 2.8.0 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Searching for docutils==0.15.2 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Best match: docutils 0.15.2 2019-11-29T12:16:00.984 INFO:teuthology.orchestra.run.smithi069.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-11-29T12:16:00.985 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T12:16:00.985 INFO:teuthology.orchestra.run.smithi069.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-29T12:16:00.985 INFO:teuthology.orchestra.run.smithi069.stdout:Finished processing dependencies for s3tests==0.0.1 2019-11-29T12:16:00.991 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-11-29T12:16:01.072 INFO:tasks.s3tests:Configuring boto... 2019-11-29T12:16:01.072 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/boto.cfg 2019-11-29T12:16:01.193 INFO:teuthology.orchestra.run.smithi069:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-11-29T12:16:04.964 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-11-29T12:16:07.986 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-11-29T12:16:10.960 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-11-29T12:16:13.119 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-11-29T12:16:15.650 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-11-29T12:16:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-11-29T12:16:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-11-29T12:16:25.910 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-11-29T12:16:26.102 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-11-29T12:16:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-11-29T12:16:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-11-29T12:16:31.713 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-11-29T12:16:34.008 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-11-29T12:16:36.509 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-11-29T12:16:36.526 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-11-29T12:16:36.602 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-11-29T12:16:36.602 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-11-29T12:16:36.602 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-11-29T12:16:36.602 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-11-29T12:16:36.602 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-11-29T12:16:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-11-29T12:16:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-11-29T12:16:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-11-29T12:16:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-11-29T12:16:36.605 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-11-29T12:16:36.610 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-11-29T12:16:36.610 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-11-29T12:16:36.610 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-11-29T12:16:36.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-11-29T12:16:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-11-29T12:16:36.613 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-11-29T12:16:36.614 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-11-29T12:16:43.586 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-11-29T12:16:43.586 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-11-29T12:16:43.587 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-11-29T12:16:43.588 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-11-29T12:16:50.689 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-11-29T12:16:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-11-29T12:17:01.954 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-11-29T12:17:05.142 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-11-29T12:17:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-11-29T12:17:15.297 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-11-29T12:17:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-11-29T12:17:29.450 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-11-29T12:17:29.450 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-11-29T12:17:29.451 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-11-29T12:17:29.452 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-11-29T12:17:29.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-11-29T12:17:29.454 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-11-29T12:17:29.455 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-11-29T12:17:29.455 INFO:teuthology.orchestra.run.smithi069.stderr:SKIP 2019-11-29T12:17:29.461 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-11-29T12:19:25.367 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-11-29T12:19:25.367 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-11-29T12:19:25.367 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-11-29T12:19:25.367 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-11-29T12:19:25.368 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-11-29T12:19:25.368 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-11-29T12:19:25.368 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-11-29T12:19:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-11-29T12:19:25.884 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-11-29T12:19:26.220 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-11-29T12:19:26.243 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-11-29T12:19:26.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-11-29T12:19:26.263 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-11-29T12:19:26.263 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-11-29T12:19:26.263 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-11-29T12:19:26.263 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-11-29T12:19:26.263 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-11-29T12:19:26.267 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-11-29T12:19:26.267 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-11-29T12:19:26.267 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-11-29T12:19:26.267 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-11-29T12:19:26.271 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-11-29T12:19:26.271 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-11-29T12:19:26.272 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-11-29T12:19:26.272 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-11-29T12:19:26.295 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-11-29T12:19:26.318 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-11-29T12:19:26.318 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-11-29T12:19:26.492 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-11-29T12:19:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok 2019-11-29T12:19:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... ok 2019-11-29T12:19:49.829 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... ok 2019-11-29T12:19:54.085 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... ok 2019-11-29T12:19:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... ok 2019-11-29T12:20:05.318 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... ok 2019-11-29T12:20:10.997 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... ok 2019-11-29T12:20:13.405 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... ok 2019-11-29T12:20:18.251 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok 2019-11-29T12:20:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... ok 2019-11-29T12:20:26.895 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... ok 2019-11-29T12:20:29.167 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... ok 2019-11-29T12:20:29.254 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:07:42.144 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.293 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:07:42.183 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.303 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:07:42.193 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.320 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:07:42.210 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.354 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:07:42.126 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.355 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:07:42.130 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.475 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:07:42.366 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.484 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:07:42.374 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:29.543 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:07:42.317 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:20:31.050 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... ok 2019-11-29T12:20:34.636 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... ok 2019-11-29T12:20:34.679 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... ok 2019-11-29T12:20:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... ok 2019-11-29T12:20:42.797 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... ok 2019-11-29T12:20:46.810 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... ok 2019-11-29T12:20:51.048 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... ok 2019-11-29T12:20:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... ok 2019-11-29T12:20:56.617 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... ok 2019-11-29T12:20:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... ok 2019-11-29T12:21:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... ok 2019-11-29T12:21:03.649 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... ok 2019-11-29T12:21:05.485 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:08:18.375 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.496 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:08:18.387 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.500 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:08:18.390 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.504 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:08:18.394 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.508 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:08:18.283 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.513 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:08:18.288 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.543 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:08:18.433 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.552 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:08:18.441 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.572 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... ok 2019-11-29T12:21:05.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... ok 2019-11-29T12:21:05.624 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:08:18.398 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:21:05.717 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... ok 2019-11-29T12:21:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... ok 2019-11-29T12:21:05.855 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... ok 2019-11-29T12:21:05.893 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... ok 2019-11-29T12:21:09.553 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... ok 2019-11-29T12:21:16.151 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... ok 2019-11-29T12:21:23.399 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... ok 2019-11-29T12:21:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... ok 2019-11-29T12:21:38.691 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... ok 2019-11-29T12:21:46.653 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... ok 2019-11-29T12:21:54.959 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... ok 2019-11-29T12:22:09.469 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... ok 2019-11-29T12:22:27.048 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... ok 2019-11-29T12:22:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... ok 2019-11-29T12:22:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... ok 2019-11-29T12:22:42.495 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... ok 2019-11-29T12:22:50.344 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... ok 2019-11-29T12:23:00.617 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... ok 2019-11-29T12:23:11.942 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... ok 2019-11-29T12:23:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... ok 2019-11-29T12:23:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... ok 2019-11-29T12:23:37.312 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... ok 2019-11-29T12:23:44.609 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... ok 2019-11-29T12:23:52.875 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... ok 2019-11-29T12:24:02.134 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... ok 2019-11-29T12:24:10.250 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... ok 2019-11-29T12:24:17.559 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... ok 2019-11-29T12:24:25.749 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... ok 2019-11-29T12:24:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... ok 2019-11-29T12:24:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... ok 2019-11-29T12:24:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... ok 2019-11-29T12:24:50.783 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... ok 2019-11-29T12:24:55.616 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... ok 2019-11-29T12:24:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... ok 2019-11-29T12:25:04.402 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... ok 2019-11-29T12:25:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... ok 2019-11-29T12:26:16.919 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:13:29.809 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:16.935 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:13:29.826 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:16.942 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:13:29.832 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:16.943 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:13:29.717 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:16.944 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:13:29.834 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:16.949 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:13:29.723 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:17.158 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:13:29.932 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:17.202 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:13:30.093 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:26:17.245 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:13:30.135 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.608 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:16:08.498 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.631 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:16:08.522 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.678 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:16:08.451 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.680 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:16:08.569 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.701 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:16:08.572 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.702 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:16:08.476 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.721 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:16:08.611 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.774 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:16:08.664 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:28:55.814 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:16:08.588 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.128 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:18:42.018 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.141 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:18:42.031 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.149 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:18:42.038 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.154 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:18:41.928 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.157 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:18:42.047 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.159 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:18:41.933 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.242 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:18:42.132 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.272 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:18:42.161 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:31:29.296 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:18:42.070 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.900 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:23:55.790 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.905 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:23:55.795 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.913 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:23:55.802 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.915 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:23:55.689 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.915 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:23:55.805 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:42.920 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:23:55.694 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:43.101 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:23:55.990 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:43.105 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:23:55.996 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:36:43.121 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:23:55.895 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:21.883 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:24:34.773 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:21.934 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:24:34.824 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:21.974 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:24:34.864 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:21.977 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:24:34.866 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:21.993 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:24:34.882 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:22.014 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:24:34.788 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:22.024 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:24:34.797 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:22.032 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:24:34.922 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:37:22.070 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:24:34.843 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:49.465 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... ok 2019-11-29T12:41:53.990 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:29:06.880 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:53.992 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:29:06.883 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.014 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:29:06.788 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.022 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:29:06.796 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.027 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:29:06.917 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.032 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:29:06.922 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.178 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:29:07.069 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.276 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:29:07.049 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:54.298 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:29:07.187 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:41:56.873 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... ok 2019-11-29T12:42:04.616 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... ok 2019-11-29T12:42:10.837 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... ok 2019-11-29T12:42:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... ok 2019-11-29T12:42:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... ok 2019-11-29T12:42:32.122 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... ok 2019-11-29T12:42:37.581 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... ok 2019-11-29T12:42:44.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... ok 2019-11-29T12:42:51.018 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... ok 2019-11-29T12:42:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... ok 2019-11-29T12:43:04.015 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... ok 2019-11-29T12:43:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... ok 2019-11-29T12:43:17.207 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... ok 2019-11-29T12:43:25.011 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... ok 2019-11-29T12:43:33.548 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... ok 2019-11-29T12:43:40.313 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... ok 2019-11-29T12:43:47.169 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... ok 2019-11-29T12:43:54.558 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... ok 2019-11-29T12:44:01.630 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... ok 2019-11-29T12:44:09.684 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... ok 2019-11-29T12:44:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-11-29T12:44:27.471 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-11-29T12:44:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... ok 2019-11-29T12:44:44.136 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... ok 2019-11-29T12:44:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... ok 2019-11-29T12:44:56.480 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... ok 2019-11-29T12:45:01.762 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... ok 2019-11-29T12:45:09.786 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... ok 2019-11-29T12:45:43.305 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:32:56.195 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.322 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:32:56.212 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.353 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:32:56.242 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.357 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:32:56.246 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.403 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:32:56.177 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.423 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:32:56.197 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.461 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:32:56.345 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.542 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:32:56.431 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:43.621 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:32:56.387 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:45:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... ok 2019-11-29T12:46:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... ok 2019-11-29T12:46:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... ok 2019-11-29T12:46:21.735 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... ok 2019-11-29T12:46:29.310 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... ok 2019-11-29T12:46:35.242 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... ok 2019-11-29T12:46:42.196 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... ok 2019-11-29T12:46:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... ok 2019-11-29T12:46:59.967 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... ok 2019-11-29T12:47:05.644 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:34:18.534 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.651 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:34:18.542 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.660 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:34:18.549 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.663 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:34:18.553 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.669 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:34:18.443 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.675 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:34:18.449 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.716 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:34:18.606 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.727 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:34:18.616 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:05.819 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:34:18.593 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:47:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... ok 2019-11-29T12:47:14.403 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... ok 2019-11-29T12:47:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... ok 2019-11-29T12:47:29.560 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... ok 2019-11-29T12:47:50.695 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... ok 2019-11-29T12:47:55.171 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... ok 2019-11-29T12:47:59.877 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... ok 2019-11-29T12:48:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... ok 2019-11-29T12:48:06.550 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... ok 2019-11-29T12:48:07.216 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... ok 2019-11-29T12:48:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... ok 2019-11-29T12:48:08.258 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... ok 2019-11-29T12:48:12.385 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... ok 2019-11-29T12:48:19.081 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... ok 2019-11-29T12:48:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... ok 2019-11-29T12:48:27.132 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... ok 2019-11-29T12:48:30.961 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... ok 2019-11-29T12:48:33.789 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... ok 2019-11-29T12:48:46.055 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... ok 2019-11-29T12:49:00.937 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... ok 2019-11-29T12:49:04.253 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... ok 2019-11-29T12:49:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... ok 2019-11-29T12:49:12.902 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... ok 2019-11-29T12:49:15.650 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... ok 2019-11-29T12:49:22.354 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... ok 2019-11-29T12:49:26.546 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... ok 2019-11-29T12:49:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... ok 2019-11-29T12:49:35.865 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... ok 2019-11-29T12:49:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... ok 2019-11-29T12:49:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... ok 2019-11-29T12:49:53.477 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... ok 2019-11-29T12:49:59.091 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... ok 2019-11-29T12:50:02.817 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... ok 2019-11-29T12:50:07.380 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... ok 2019-11-29T12:50:10.422 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... ok 2019-11-29T12:50:14.717 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... ok 2019-11-29T12:50:19.236 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... ok 2019-11-29T12:50:24.742 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... ok 2019-11-29T12:50:29.757 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... ok 2019-11-29T12:50:33.561 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... ok 2019-11-29T12:50:37.136 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... ok 2019-11-29T12:50:42.627 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... ok 2019-11-29T12:50:48.635 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... ok 2019-11-29T12:50:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... ok 2019-11-29T12:50:53.795 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... ok 2019-11-29T12:50:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... ok 2019-11-29T12:50:58.711 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... ok 2019-11-29T12:51:02.247 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... ok 2019-11-29T12:51:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... ok 2019-11-29T12:51:09.306 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... ok 2019-11-29T12:51:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... ok 2019-11-29T12:51:16.203 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... ok 2019-11-29T12:51:19.729 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... ok 2019-11-29T12:51:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... ok 2019-11-29T12:51:23.956 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... ok 2019-11-29T12:51:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... ok 2019-11-29T12:51:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... ok 2019-11-29T12:51:33.903 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... ok 2019-11-29T12:51:37.600 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... ok 2019-11-29T12:51:40.490 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... ok 2019-11-29T12:51:43.507 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... ok 2019-11-29T12:51:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... ok 2019-11-29T12:51:51.264 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... ok 2019-11-29T12:51:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... ok 2019-11-29T12:52:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... ok 2019-11-29T12:52:05.078 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... ok 2019-11-29T12:52:10.411 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... ok 2019-11-29T12:52:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... ok 2019-11-29T12:52:19.491 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:39:32.381 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.502 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:39:32.393 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.517 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:39:32.406 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.524 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:39:32.298 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.525 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:39:32.415 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.553 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:39:32.327 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.696 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:39:32.586 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.722 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:39:32.611 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:19.793 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:39:32.567 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:52:20.612 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... ok 2019-11-29T12:52:24.924 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... ok 2019-11-29T12:52:29.927 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... ok 2019-11-29T12:52:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... ok 2019-11-29T12:52:41.327 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok 2019-11-29T12:52:43.909 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... ok 2019-11-29T12:52:48.658 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... ok 2019-11-29T12:52:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... ok 2019-11-29T12:52:57.743 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok 2019-11-29T12:53:01.749 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... ok 2019-11-29T12:53:06.755 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... ok 2019-11-29T12:53:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... ok 2019-11-29T12:53:26.881 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... ok 2019-11-29T12:53:31.983 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... ok 2019-11-29T12:53:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... ok 2019-11-29T12:53:45.967 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... ok 2019-11-29T12:53:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... ok 2019-11-29T12:53:55.275 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... ok 2019-11-29T12:53:59.870 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... ok 2019-11-29T12:54:04.354 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... ok 2019-11-29T12:54:04.753 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:17.643 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.807 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:17.696 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:17.697 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.810 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:17.700 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.842 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:17.615 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.854 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:17.628 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.867 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:17.757 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.899 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:17.789 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:04.947 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:17.720 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:06.729 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... ok 2019-11-29T12:54:10.354 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... ok 2019-11-29T12:54:15.049 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:27.939 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.060 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:27.950 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.085 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:41:27.975 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.094 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:27.868 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.103 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:27.993 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.113 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:28.003 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.128 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:27.901 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.141 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:41:28.031 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:15.154 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:41:27.928 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:54:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... ok 2019-11-29T12:54:23.330 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... ok 2019-11-29T12:54:27.639 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... ok 2019-11-29T12:54:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... ok 2019-11-29T12:54:33.990 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... ok 2019-11-29T12:54:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... ok 2019-11-29T12:54:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... ok 2019-11-29T12:54:49.126 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... ok 2019-11-29T12:54:53.033 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... ok 2019-11-29T12:54:55.352 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... ok 2019-11-29T12:54:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... ok 2019-11-29T12:54:57.106 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... ok 2019-11-29T12:54:57.474 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... ok 2019-11-29T12:54:57.899 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... ok 2019-11-29T12:55:02.455 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... ok 2019-11-29T12:55:05.748 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... ok 2019-11-29T12:55:08.714 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... ok 2019-11-29T12:55:10.275 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... ok 2019-11-29T12:55:12.949 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... ok 2019-11-29T12:55:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... ok 2019-11-29T12:55:18.832 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... ok 2019-11-29T12:55:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... ok 2019-11-29T12:55:23.897 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... ok 2019-11-29T12:55:27.208 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... ok 2019-11-29T12:55:32.922 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... ok 2019-11-29T12:55:36.788 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... ok 2019-11-29T12:55:39.889 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... ok 2019-11-29T12:55:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... ok 2019-11-29T12:55:48.557 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... ok 2019-11-29T12:55:54.026 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... ok 2019-11-29T12:55:58.221 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... ok 2019-11-29T12:56:01.772 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... ok 2019-11-29T12:56:06.317 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... ok 2019-11-29T12:56:11.447 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... ok 2019-11-29T12:56:15.746 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... ok 2019-11-29T12:56:20.985 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... ok 2019-11-29T12:56:25.736 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... ok 2019-11-29T12:56:31.925 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... ok 2019-11-29T12:56:37.535 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... ok 2019-11-29T12:56:42.251 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... ok 2019-11-29T12:56:47.540 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... ok 2019-11-29T12:56:53.907 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... ok 2019-11-29T12:56:56.458 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... ok 2019-11-29T12:57:02.919 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... ok 2019-11-29T12:57:08.062 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... ok 2019-11-29T12:57:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... ok 2019-11-29T12:57:16.475 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... ok 2019-11-29T12:57:18.817 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... ok 2019-11-29T12:57:26.033 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... ok 2019-11-29T12:57:30.053 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:44:42.944 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.064 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:44:42.955 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.077 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:44:42.967 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.080 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:44:42.970 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.083 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:44:42.857 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.095 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:44:42.868 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.206 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:44:43.096 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.253 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:44:43.142 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:30.282 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:44:43.056 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T12:57:34.131 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... ok 2019-11-29T12:57:43.870 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... ok 2019-11-29T12:57:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... ok 2019-11-29T12:58:01.324 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... ok 2019-11-29T12:58:10.678 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... ok 2019-11-29T12:58:19.949 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... ok 2019-11-29T12:58:30.038 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... ok 2019-11-29T12:58:39.498 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... ok 2019-11-29T12:58:49.259 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... ok 2019-11-29T12:59:01.531 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... ok 2019-11-29T12:59:13.150 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... ok 2019-11-29T12:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... ok 2019-11-29T12:59:38.059 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... ok 2019-11-29T12:59:38.499 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... ok 2019-11-29T12:59:39.485 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... ok 2019-11-29T12:59:39.592 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... ok 2019-11-29T12:59:51.461 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... ok 2019-11-29T13:00:00.965 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok 2019-11-29T13:00:03.162 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok 2019-11-29T13:00:04.940 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok 2019-11-29T13:00:12.148 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... ok 2019-11-29T13:00:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... ok 2019-11-29T13:01:00.910 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... ok 2019-11-29T13:01:08.959 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... ok 2019-11-29T13:01:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... ok 2019-11-29T13:01:21.923 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... ok 2019-11-29T13:01:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... ok 2019-11-29T13:01:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok 2019-11-29T13:01:42.336 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... ok 2019-11-29T13:01:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... ok 2019-11-29T13:01:55.591 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... ok 2019-11-29T13:02:05.409 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... ok 2019-11-29T13:02:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... ok 2019-11-29T13:02:26.619 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:39.509 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.625 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:39.515 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.685 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:39.575 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.703 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:39.592 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.714 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:39.487 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.715 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:39.489 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.821 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:39.710 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.847 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:39.737 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:26.920 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:39.693 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... ok 2019-11-29T13:02:32.388 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... ok 2019-11-29T13:02:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... ok 2019-11-29T13:02:41.771 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:54.661 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.779 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:54.670 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.785 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:54.674 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.790 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:54.680 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.814 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:54.587 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.814 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:49:54.704 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.836 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:49:54.726 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.841 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:54.614 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:02:41.894 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:49:54.668 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:03:13.117 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... ok 2019-11-29T13:03:26.020 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... ok 2019-11-29T13:04:12.000 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:04:11.986+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:04:45.789 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... ok 2019-11-29T13:04:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... ok 2019-11-29T13:04:57.103 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... ok 2019-11-29T13:05:13.645 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:05:13.605+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:05:14.022 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... ok 2019-11-29T13:05:19.097 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... ok 2019-11-29T13:05:34.120 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... ok 2019-11-29T13:05:49.089 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:05:49.085+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:06:01.199 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:06:01.174+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:06:13.353 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:06:13.344+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:06:31.718 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... ok 2019-11-29T13:07:48.576 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... ok 2019-11-29T13:07:53.722 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:55:06.612 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.728 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:55:06.616 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.734 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:55:06.624 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.741 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:55:06.631 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.744 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:55:06.517 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.749 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:55:06.523 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.914 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:55:06.804 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:53.933 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:55:06.823 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:07:54.019 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:55:06.793 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:08:35.928 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:08:35.810+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:08:57.197 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:08:57.175+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:08:58.709 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:08:58.610+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:09:14.291 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:09:14.285+0000 40a60700 -1 handle_data failed with exit code -5 2019-11-29T13:09:16.334 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:09:16.324+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:09:17.030 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... ok 2019-11-29T13:09:17.109 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:09:17.073+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:09:54.924 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:09:54.913+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:10:25.227 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:10:25.125+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:10:48.110 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:58:00.998 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.141 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:58:01.031 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.183 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:58:01.073 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.186 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:58:01.076 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.212 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:58:00.983 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.213 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:58:00.986 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.262 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:01:58:01.151 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.281 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:01:58:01.170 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:10:48.377 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:01:58:01.150 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:12:25.669 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:12:25.611+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:12:55.439 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:12:55.390+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:13:03.812 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:00:16.703 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.823 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:00:16.714 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.833 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:00:16.723 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.839 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:00:16.729 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.845 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:00:16.618 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.858 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:00:16.631 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.927 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:00:16.816 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:03.932 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:00:16.823 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:04.031 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:00:16.804 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:13:44.937 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:13:44.909+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:14:18.809 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:14:18.753+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:16:02.525 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... ok 2019-11-29T13:17:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... ok 2019-11-29T13:17:37.675 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:17:37.550+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:17:51.843 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:17:51.751+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:18:07.942 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:18:07.917+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:18:15.537 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:05:28.427 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.544 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:05:28.434 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.555 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:05:28.445 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.558 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:05:28.448 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.565 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:05:28.339 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.581 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:05:28.355 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.719 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:05:28.610 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.748 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:05:28.637 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:15.907 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:05:28.681 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:18:24.611 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:18:24.566+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:18:46.568 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:18:46.490+0000 3fa5e700 -1 handle_data failed with exit code -5 2019-11-29T13:19:03.270 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T13:19:03.263+0000 4025f700 -1 handle_data failed with exit code -5 2019-11-29T13:19:04.444 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... ok 2019-11-29T13:19:09.747 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:06:22.637 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.758 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:06:22.648 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.759 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:06:22.649 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.760 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:06:22.534 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.761 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:06:22.535 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.770 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:06:22.660 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.795 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:06:22.685 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.798 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:06:22.572 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:09.807 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:06:22.698 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:19:29.041 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... ok 2019-11-29T13:19:55.583 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... ok 2019-11-29T13:20:14.608 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok 2019-11-29T13:20:27.733 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... ok 2019-11-29T13:20:31.997 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... ok 2019-11-29T13:20:56.944 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... ok 2019-11-29T13:21:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... ok 2019-11-29T13:21:11.076 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... ok 2019-11-29T13:21:14.621 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... ok 2019-11-29T13:21:19.398 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... ok 2019-11-29T13:21:42.376 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... ok 2019-11-29T13:21:50.305 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... ok 2019-11-29T13:21:56.877 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... ok 2019-11-29T13:22:05.715 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... ok 2019-11-29T13:22:15.137 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... ok 2019-11-29T13:22:31.060 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... ok 2019-11-29T13:22:41.011 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... ok 2019-11-29T13:22:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... ok 2019-11-29T13:23:16.627 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... ok 2019-11-29T13:23:26.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:10:39.698 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:26.816 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:10:39.705 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:26.832 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:10:39.722 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:26.836 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:10:39.610 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:26.837 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:10:39.726 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:26.841 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:10:39.614 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:27.018 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:10:39.908 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:27.037 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:10:39.926 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:27.050 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:10:39.824 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:23:30.168 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... ok 2019-11-29T13:23:48.126 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... ok 2019-11-29T13:24:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... ok 2019-11-29T13:24:21.563 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... ok 2019-11-29T13:24:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... ok 2019-11-29T13:24:48.606 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... ok 2019-11-29T13:24:52.191 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... ok 2019-11-29T13:25:08.863 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... ok 2019-11-29T13:25:13.422 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... ok 2019-11-29T13:25:15.936 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... ok 2019-11-29T13:25:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... ok 2019-11-29T13:25:21.537 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... ok 2019-11-29T13:25:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... ok 2019-11-29T13:27:31.110 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:14:43.999 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.113 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:14:44.004 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.131 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:14:44.018 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.150 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:14:43.923 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.152 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:14:43.927 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.156 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:14:44.043 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.234 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:14:44.125 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.271 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:14:44.045 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:27:31.276 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:14:44.165 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.027 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:15:51.918 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.034 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:15:51.925 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.044 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:15:51.934 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.048 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:15:51.938 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.052 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:15:51.825 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.070 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:15:51.845 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.109 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:15:51.999 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.114 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:15:52.005 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:28:39.139 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:15:51.913 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:30:32.300 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... ok 2019-11-29T13:31:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... ok 2019-11-29T13:31:43.008 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... ok 2019-11-29T13:32:01.388 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... ok 2019-11-29T13:32:18.313 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... ok 2019-11-29T13:33:49.155 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:21:02.044 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.165 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:21:02.055 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.167 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:21:02.057 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.169 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:21:01.943 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.171 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:21:02.061 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.175 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:21:01.949 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.372 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:21:02.147 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.386 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:21:02.277 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:33:49.440 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:21:02.330 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:29.415 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... ok 2019-11-29T13:34:57.449 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:22:10.340 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.479 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:22:10.368 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.504 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:22:10.393 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.508 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:22:10.398 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.530 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:22:10.305 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.531 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:22:10.422 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.542 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:22:10.316 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.565 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:22:10.454 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:34:57.653 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:22:10.427 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.765 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:23:10.655 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.772 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:23:10.662 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.782 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:23:10.672 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.813 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:23:10.703 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.822 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:23:10.596 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.829 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:23:10.719 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.845 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:23:10.618 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.871 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:23:10.761 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:35:57.931 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:23:10.705 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:36:01.829 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... ok 2019-11-29T13:38:59.386 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:26:12.276 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.398 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:26:12.289 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.406 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:26:12.296 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.410 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:26:12.300 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.413 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:26:12.187 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.417 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:26:12.191 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.566 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:26:12.340 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.569 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:26:12.460 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:38:59.571 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:26:12.461 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:43:26.650 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... ok 2019-11-29T13:44:10.208 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:23.099 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.214 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:23.105 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.225 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:23.116 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.231 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:23.121 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.234 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:23.008 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.241 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:23.015 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.414 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:23.305 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.418 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:23.308 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:10.495 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:23.269 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.065 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:41.955 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.074 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:41.965 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.103 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:31:41.993 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.104 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:41.994 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.109 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:41.999 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.128 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:41.902 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.132 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:41.906 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.135 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:31:42.025 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:44:29.166 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:31:41.940 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:45:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... ok 2019-11-29T13:45:37.421 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... ok 2019-11-29T13:46:17.301 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... ok 2019-11-29T13:46:45.937 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... ok 2019-11-29T13:47:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... ok 2019-11-29T13:47:26.060 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... ok 2019-11-29T13:47:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... ok 2019-11-29T13:48:07.850 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... ok 2019-11-29T13:48:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... Exception in thread Thread-52: 2019-11-29T13:48:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:15.322 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:15.323 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:15.323 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:15.323 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:15.323 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:15.323 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:15.330 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-54: 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:15.331 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:15.332 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:15.477 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-55: 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:15.478 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:15.479 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-53: 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:15.572 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:15.573 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:34.026 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-65: 2019-11-29T13:48:34.026 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:34.026 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:34.026 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:34.027 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:34.028 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:34.028 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:34.028 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-64: 2019-11-29T13:48:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:34.594 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:34.595 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:34.596 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:34.597 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:34.636 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-63: 2019-11-29T13:48:34.636 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:34.637 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:34.638 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:34.639 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:34.639 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-62: 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:34.889 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:34.891 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-75: 2019-11-29T13:48:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:51.962 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:51.962 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:51.962 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:51.963 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:51.964 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:51.964 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-73: 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:52.169 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:52.170 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:52.171 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-74: 2019-11-29T13:48:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:52.769 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:52.770 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:52.771 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:52.771 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:52.771 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:48:53.162 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-71: 2019-11-29T13:48:53.162 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:48:53.162 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:48:53.163 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:48:53.164 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:48:53.164 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:48:53.164 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:48:53.164 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:12.288 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-83: 2019-11-29T13:49:12.288 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:12.288 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:12.288 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:12.288 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:12.289 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:12.290 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:12.290 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:12.314 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-81: 2019-11-29T13:49:12.314 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:12.314 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:12.314 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:12.314 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:12.315 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:12.316 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:12.316 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:12.316 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-84: 2019-11-29T13:49:12.316 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:12.316 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:12.317 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:12.318 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:12.318 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:12.318 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:12.318 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:12.586 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-85: 2019-11-29T13:49:12.586 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:12.587 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:12.588 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:21.634 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:36:34.525 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.641 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:36:34.532 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.648 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:36:34.538 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.653 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:36:34.543 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.657 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:36:34.431 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.661 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:36:34.435 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:36:34.698 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.820 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:36:34.710 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:21.857 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:36:34.631 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:49:31.439 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-92: 2019-11-29T13:49:31.440 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:31.440 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:31.440 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:31.440 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:31.441 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:31.441 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:31.441 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:31.441 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:31.441 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:31.442 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:31.442 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:31.442 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-93: 2019-11-29T13:49:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:31.463 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:31.464 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:31.465 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:31.465 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:31.465 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:31.465 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:31.466 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-95: 2019-11-29T13:49:31.466 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:31.466 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:31.466 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:31.467 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:31.467 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:31.467 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:31.467 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:31.467 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:31.468 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:31.468 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:31.468 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:31.468 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:31.469 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:31.469 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-94: 2019-11-29T13:49:31.469 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:31.469 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:31.470 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:31.470 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:31.470 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:31.470 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:31.471 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:31.471 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:31.471 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:31.472 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:31.472 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:31.472 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:31.473 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:43.689 INFO:teuthology.orchestra.run.smithi069.stderr:ok 2019-11-29T13:49:52.864 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... Exception in thread Thread-102: 2019-11-29T13:49:52.864 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:52.865 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:52.865 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:52.865 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:52.865 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:52.865 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:52.866 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:52.866 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:52.866 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:52.866 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:52.867 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:52.867 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:52.867 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:52.867 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-103: 2019-11-29T13:49:52.867 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:52.868 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:52.868 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:52.868 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:52.868 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:52.869 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:52.869 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:52.869 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:52.869 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:52.870 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:52.870 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:52.870 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:52.870 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-104: 2019-11-29T13:49:52.921 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:52.921 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:52.921 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:52.921 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:52.922 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:52.922 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:52.922 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:52.922 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:52.922 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:52.923 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:52.923 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:52.923 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:53.251 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-105: 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:53.252 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:53.253 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-106: 2019-11-29T13:49:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:54.399 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:54.399 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:54.399 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:54.400 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:54.401 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:54.401 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:54.671 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-107: 2019-11-29T13:49:54.671 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:54.672 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:54.672 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:54.672 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:54.672 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:54.672 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:54.673 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:55.416 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-110: 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8578, in _do_remove_ver 2019-11-29T13:49:55.417 INFO:teuthology.orchestra.run.smithi069.stderr: client.delete_object(Bucket=bucket_name, Key=key, VersionId=version_id) 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the DeleteObject operation: Unknown 2019-11-29T13:49:55.418 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-108: 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:55.419 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:55.420 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:Exception in thread Thread-109: 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:Traceback (most recent call last): 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr: self.run() 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-11-29T13:49:55.475 INFO:teuthology.orchestra.run.smithi069.stderr: self.__target(*self.__args, **self.__kwargs) 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: return self._make_api_call(operation_name, kwargs) 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call 2019-11-29T13:49:55.476 INFO:teuthology.orchestra.run.smithi069.stderr: raise error_class(parsed_response, operation_name) 2019-11-29T13:49:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-11-29T13:49:55.477 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T13:50:19.916 INFO:teuthology.orchestra.run.smithi069.stderr:ok 2019-11-29T13:50:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... ok 2019-11-29T13:50:30.065 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... ok 2019-11-29T13:50:35.281 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... ok 2019-11-29T13:50:37.401 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... ok 2019-11-29T13:50:40.077 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... ok 2019-11-29T13:50:43.453 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... ok 2019-11-29T13:50:47.538 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... ok 2019-11-29T13:50:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... ok 2019-11-29T13:50:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:Check for valid x-amz-expiration header after PUT ... ok 2019-11-29T13:51:01.390 INFO:teuthology.orchestra.run.smithi069.stderr:Check for valid x-amz-expiration header on HEAD request ... ok 2019-11-29T13:51:07.928 INFO:teuthology.orchestra.run.smithi069.stderr:Check for valid x-amz-expiration header on GET request ... ok 2019-11-29T13:51:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:Check x-amz-expiration-header object tags interaction ... ok 2019-11-29T13:51:23.262 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... ok 2019-11-29T13:51:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... ok 2019-11-29T13:51:31.619 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... ok 2019-11-29T13:51:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... ok 2019-11-29T13:51:41.006 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... ok 2019-11-29T13:51:46.555 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... ok 2019-11-29T13:51:52.327 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... ok 2019-11-29T13:51:57.272 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... ok 2019-11-29T13:52:00.623 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... ok 2019-11-29T13:52:06.982 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok 2019-11-29T13:52:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok 2019-11-29T13:52:13.322 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok 2019-11-29T13:52:15.400 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok 2019-11-29T13:52:18.670 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok 2019-11-29T13:52:21.537 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok 2019-11-29T13:52:26.539 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok 2019-11-29T13:52:50.497 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:40:03.387 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.508 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:40:03.399 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.545 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:40:03.435 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.584 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:40:03.473 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.587 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:40:03.361 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.591 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:40:03.365 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.600 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:40:03.491 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.706 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:40:03.596 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:52:50.732 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:40:03.506 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:53:52.583 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok 2019-11-29T13:54:33.114 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:41:46.005 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.117 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:41:46.008 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.129 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:41:46.019 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.136 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:41:46.026 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.138 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:41:45.913 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.143 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:41:45.917 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.204 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:41:46.095 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.225 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:41:46.115 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.274 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:41:46.049 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:54:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok 2019-11-29T13:54:38.682 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok 2019-11-29T13:54:43.850 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... ok 2019-11-29T13:54:47.470 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... ok 2019-11-29T13:54:50.152 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... ok 2019-11-29T13:54:52.532 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... ok 2019-11-29T13:56:11.529 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... ok 2019-11-29T13:56:42.889 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... ok 2019-11-29T13:57:14.291 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... ok 2019-11-29T13:57:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... ok 2019-11-29T13:57:23.009 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... ok 2019-11-29T13:57:28.137 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... ok 2019-11-29T13:57:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... ok 2019-11-29T13:57:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... ok 2019-11-29T13:57:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... ok 2019-11-29T13:57:50.021 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... ok 2019-11-29T13:57:55.213 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... ok 2019-11-29T13:58:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... ok 2019-11-29T13:58:13.282 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... ok 2019-11-29T13:58:24.760 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... ok 2019-11-29T13:58:34.817 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... ok 2019-11-29T13:58:42.869 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... ok 2019-11-29T13:58:51.444 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... ok 2019-11-29T13:59:00.067 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... ok 2019-11-29T13:59:06.050 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... ok 2019-11-29T13:59:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... ok 2019-11-29T13:59:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... ok 2019-11-29T13:59:28.019 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... ok 2019-11-29T13:59:39.521 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... ok 2019-11-29T13:59:45.925 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:46:58.815 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:45.927 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:46:58.818 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:45.958 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:46:58.845 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:45.961 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:46:58.851 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:45.965 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:46:58.739 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:45.972 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:46:58.747 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:46.177 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:46:59.060 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:46.181 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:46:59.072 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:46.290 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:46:59.064 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T13:59:47.183 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... ok 2019-11-29T13:59:51.840 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... ok 2019-11-29T13:59:55.498 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... ok 2019-11-29T13:59:58.606 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... ok 2019-11-29T14:00:07.411 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... ok 2019-11-29T14:00:16.844 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... ok 2019-11-29T14:00:27.828 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... ok 2019-11-29T14:00:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... ok 2019-11-29T14:01:11.917 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:48:24.807 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:11.938 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:48:24.829 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:11.963 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:48:24.853 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:11.966 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:48:24.856 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:11.990 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:48:24.881 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:12.013 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:48:24.786 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:12.017 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:48:24.792 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:12.043 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:48:24.933 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:01:12.097 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:48:24.866 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:02:11.103 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... ok 2019-11-29T14:02:23.087 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... ok 2019-11-29T14:02:38.090 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... ok 2019-11-29T14:02:52.382 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... ok 2019-11-29T14:03:10.594 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... ok 2019-11-29T14:03:22.932 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... ok 2019-11-29T14:03:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... ok 2019-11-29T14:03:37.708 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... ok 2019-11-29T14:03:46.511 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... ok 2019-11-29T14:03:52.196 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... ok 2019-11-29T14:03:54.385 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... ok 2019-11-29T14:03:56.769 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... ok 2019-11-29T14:03:58.391 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... ok 2019-11-29T14:04:01.117 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... ok 2019-11-29T14:04:04.212 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... ok 2019-11-29T14:04:07.339 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... ok 2019-11-29T14:04:11.585 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... ok 2019-11-29T14:04:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... ok 2019-11-29T14:04:25.680 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... ok 2019-11-29T14:04:29.024 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... ok 2019-11-29T14:04:35.686 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... ok 2019-11-29T14:04:49.186 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... ok 2019-11-29T14:04:52.315 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... ok 2019-11-29T14:04:57.509 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:52:10.400 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.512 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:52:10.403 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.524 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:52:10.414 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.526 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:52:10.416 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.529 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:52:10.303 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.545 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:52:10.320 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.691 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:52:10.581 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.730 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:52:10.618 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:04:57.759 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:52:10.533 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:05:01.726 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... ok 2019-11-29T14:05:14.963 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... ok 2019-11-29T14:05:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... ok 2019-11-29T14:05:35.414 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... ok 2019-11-29T14:05:45.969 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... ok 2019-11-29T14:05:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... ok 2019-11-29T14:06:01.287 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... ok 2019-11-29T14:06:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... ok 2019-11-29T14:06:09.352 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... ok 2019-11-29T14:06:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... ok 2019-11-29T14:06:21.697 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... ok 2019-11-29T14:06:31.488 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... ok 2019-11-29T14:06:44.101 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... ok 2019-11-29T14:06:57.268 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... ok 2019-11-29T14:07:10.779 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... ok 2019-11-29T14:07:10.801 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-11-29T14:09:33.328 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:56:46.219 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.362 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:56:46.253 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.371 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:56:46.261 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.376 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:56:46.266 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.414 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:56:46.188 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.420 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:56:46.195 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.485 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:56:46.376 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.518 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:56:46.408 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:09:33.562 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:56:46.337 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.553 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:57:21.444 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.566 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:57:21.457 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.582 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:57:21.472 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.583 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:57:21.357 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.589 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:57:21.364 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.590 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:57:21.480 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.618 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:02:57:21.508 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.645 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:02:57:21.535 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:10:08.677 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:02:57:21.443 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.381 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:32.264 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.390 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:32.274 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.392 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:32.282 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.398 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:32.288 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.410 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:32.185 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.415 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:32.190 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.561 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:32.452 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.621 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:32.510 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:19.642 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:32.417 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.605 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:57.496 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.645 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:57.536 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.682 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:02:57.573 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.690 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:57.580 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.698 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:57.588 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.707 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:57.481 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.718 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:57.493 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.734 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:02:57.624 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:15:44.758 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:02:57.533 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.082 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:05:07.973 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.086 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:05:07.977 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.148 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:05:08.039 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.155 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:05:07.930 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.160 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:05:07.935 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.162 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:05:08.053 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.165 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:05:08.055 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.269 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:05:08.159 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:17:55.269 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:05:08.041 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.689 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:07:43.580 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.707 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:07:43.599 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.709 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:07:43.600 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.714 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:07:43.604 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.719 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:07:43.494 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.729 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:07:43.504 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.813 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:07:43.704 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.824 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:07:43.715 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:20:30.884 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:07:43.659 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.680 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:12:55.571 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.689 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:12:55.580 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.703 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:12:55.477 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.703 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:12:55.592 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.710 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:12:55.485 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.711 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:12:55.601 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.900 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:12:55.791 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.931 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:12:55.816 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:25:42.973 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:12:55.748 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.698 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:13:29.589 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.721 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:13:29.612 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.738 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:13:29.628 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.767 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:13:29.658 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.773 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:13:29.664 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.801 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:13:29.692 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.804 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:13:29.578 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.806 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:13:29.581 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:26:16.906 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:13:29.681 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.559 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:18:06.450 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.568 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:18:06.459 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.577 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:18:06.467 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.582 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:18:06.357 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.583 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:18:06.473 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.586 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:18:06.362 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.747 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:18:06.638 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.759 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:18:06.534 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:30:53.769 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:18:06.659 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.155 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:21:51.045 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.190 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:21:51.081 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.206 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:21:51.091 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.206 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:21:51.097 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.219 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:21:50.994 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.221 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:21:50.997 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.298 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:21:51.188 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.308 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:21:51.199 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:34:38.341 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:21:51.116 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.054 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:23:17.945 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.073 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:23:17.965 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.095 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:23:17.985 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.096 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:23:17.987 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.120 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:23:17.895 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.124 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:23:17.899 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.145 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:23:18.036 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.158 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:23:18.048 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:36:05.224 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:23:18.000 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.239 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:28:32.130 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.271 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:28:32.162 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.284 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:28:32.175 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.288 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:28:32.178 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.311 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:28:32.086 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.317 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:28:32.093 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.549 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:28:32.324 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.555 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:28:32.446 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:41:19.565 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:28:32.456 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.823 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:30:12.714 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.871 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:30:12.762 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.889 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:30:12.664 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.904 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:30:12.795 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.909 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:30:12.685 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.914 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:30:12.805 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.936 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:30:12.827 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:42:59.988 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:30:12.878 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:43:00.010 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:30:12.785 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.373 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:34.263 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.397 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:34.287 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.398 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:34.288 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.408 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:34.184 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.429 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:34.319 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.451 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:34.226 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.618 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:34.509 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.673 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:34.563 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:21.765 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:34.540 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.600 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:54.491 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.608 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:54.499 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.616 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:54.506 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.622 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:54.512 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.637 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:54.412 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.648 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:38:54.540 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.654 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:54.429 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.656 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:38:54.546 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:51:41.737 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:38:54.513 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.329 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:44:07.220 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.340 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:44:07.231 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.347 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:44:07.238 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.353 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:44:07.243 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.357 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:44:07.133 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.370 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:44:07.146 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.577 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:44:07.468 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.587 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:44:07.478 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:56:54.686 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:44:07.461 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:42.917 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:46:55.808 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:42.933 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:46:55.819 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:42.968 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:46:55.857 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:42.972 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:46:55.860 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:42.997 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:46:55.773 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:43.000 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:46:55.775 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:43.037 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:46:55.928 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:43.076 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:46:55.966 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T14:59:43.113 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:46:55.882 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.157 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:49:16.048 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.162 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:49:16.053 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.175 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:49:15.951 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.177 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:49:16.067 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.180 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:49:16.071 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.181 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:49:15.957 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.286 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:49:16.176 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.288 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:49:16.179 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:02:03.303 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:49:16.078 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.221 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:54:27.112 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.231 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:54:27.122 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.237 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:54:27.128 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.239 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:54:27.130 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.246 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:54:27.022 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.251 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:54:27.027 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.424 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:54:27.315 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.448 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:54:27.339 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:07:14.486 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:54:27.262 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.844 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:55:17.735 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.882 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:55:17.773 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.895 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:55:17.784 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.896 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:55:17.786 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.937 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:55:17.712 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.943 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:55:17.719 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.945 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:55:17.836 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:04.949 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:55:17.840 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:08:05.007 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:55:17.783 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.731 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:59:38.622 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.745 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:59:38.636 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.753 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:59:38.643 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.757 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:59:38.648 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.761 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:59:38.537 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.785 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:59:38.561 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.920 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:03:59:38.810 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:25.925 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:03:59:38.816 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:12:26.006 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:03:59:38.781 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.303 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:03:44.194 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.312 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:03:44.203 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.330 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:03:44.220 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.363 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:03:44.253 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.394 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:03:44.170 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.397 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:03:44.173 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.421 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:03:44.311 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.514 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:03:44.404 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:16:31.538 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:03:44.314 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.452 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:04:49.343 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.469 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:04:49.350 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.471 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:04:49.247 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.474 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:04:49.364 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.479 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:04:49.255 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.480 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:04:49.371 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.530 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:04:49.421 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.563 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:04:49.453 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:17:36.569 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:04:49.343 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.612 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:10:01.503 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.623 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:10:01.514 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.627 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:10:01.517 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.633 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:10:01.524 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.642 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:10:01.418 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.646 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:10:01.422 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.805 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:10:01.695 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.878 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:10:01.654 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:22:48.889 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:10:01.780 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.798 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:12:05.689 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.799 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:12:05.690 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.849 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:12:05.740 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.851 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:12:05.742 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.875 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:12:05.766 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.887 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:12:05.663 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.906 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:12:05.682 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:52.925 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:12:05.812 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:24:53.036 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:12:05.812 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:15:13.699 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.816 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:15:13.707 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.826 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:15:13.717 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.832 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:15:13.608 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.836 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:15:13.727 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.837 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:15:13.613 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.958 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:15:13.850 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.968 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:15:13.744 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:28:00.981 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:15:13.872 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.008 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:26.899 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.022 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:26.914 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.202 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:27.093 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.392 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:27.283 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.396 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:27.286 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.411 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:27.187 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.422 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:27.199 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.635 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:27.526 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:14.671 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:27.448 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:15.769 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:28.661 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:15.771 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:28.662 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:15.783 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:20:28.674 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.025 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:28.916 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.027 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:28.918 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.042 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:28.819 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.044 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:28.820 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.047 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:20:28.937 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:33:16.079 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:20:28.856 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.365 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:18.256 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.397 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:18.288 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.411 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:18.302 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.426 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:18.203 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.434 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:18.325 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.482 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:18.258 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.569 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:18.460 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.607 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:18.498 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:05.700 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:18.477 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.127 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:38.019 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.133 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:38.024 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.142 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:38.033 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.146 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:38.037 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.151 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:37.928 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.163 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:25:38.054 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.163 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:37.940 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.191 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:25:38.081 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:38:25.206 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:25:37.983 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.057 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:28:48.949 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.065 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:28:48.956 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.091 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:28:48.867 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.105 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:28:48.882 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.120 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:28:49.011 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.122 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:28:49.013 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.152 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:28:49.044 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.193 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:28:48.969 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:41:36.214 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:28:49.105 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.494 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:30:49.385 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.498 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:30:49.390 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.514 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:30:49.405 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.520 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:30:49.411 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.521 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:30:49.297 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.525 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:30:49.302 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.611 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:30:49.502 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.628 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:30:49.518 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:43:36.684 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:30:49.460 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.387 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:36:02.278 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.398 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:36:02.289 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.405 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:36:02.295 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.408 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:36:02.184 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.410 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:36:02.300 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.414 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:36:02.189 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.603 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:36:02.493 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.630 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:36:02.521 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:48:49.660 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:36:02.436 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.468 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:37:10.358 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.513 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:37:10.404 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.516 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:37:10.406 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.518 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:37:10.408 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.559 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:37:10.449 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.563 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:37:10.338 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.570 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:37:10.460 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.570 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:37:10.345 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:49:57.656 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:37:10.431 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-11-29T15:52:32.741 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-11-29T15:52:32.741 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-11-29T15:52:32.742 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-11-29T15:52:32.742 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-11-29T15:52:32.742 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-11-29T15:52:32.743 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-11-29T15:52:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-11-29T15:52:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-11-29T15:52:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-11-29T15:52:34.228 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-11-29T15:52:34.248 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-11-29T15:52:34.248 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-11-29T15:52:34.249 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-11-29T15:52:34.250 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-11-29T15:52:34.250 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-11-29T15:52:34.253 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-11-29T15:52:34.253 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-11-29T15:52:34.254 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-11-29T15:52:34.254 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-11-29T15:52:34.260 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-11-29T15:52:34.260 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-11-29T15:52:34.261 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-11-29T15:52:34.261 INFO:teuthology.orchestra.run.smithi069.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-11-29T15:52:34.279 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-11-29T15:52:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-11-29T15:52:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr:---------------------------------------------------------------------- 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr:Ran 611 tests in 12992.916s 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr: 2019-11-29T15:52:34.438 INFO:teuthology.orchestra.run.smithi069.stderr:OK (SKIP=75) 2019-11-29T15:52:34.487 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-11-29T15:52:34.488 INFO:teuthology.orchestra.run.smithi069:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-11-29T15:52:34.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-29T15:52:34.629 INFO:tasks.s3tests:Cleaning up boto... 2019-11-29T15:52:34.629 INFO:teuthology.orchestra.run.smithi069:> rm /home/ubuntu/cephtest/boto.cfg 2019-11-29T15:52:34.685 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-11-29T15:52:34.791 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.788+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.808+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.808+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:34.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.815+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:34.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.815+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.820+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.820+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:34.827+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:35.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.023+0000 7f1365f4aec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T15:52:35.025 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.023+0000 7f1365f4aec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T15:52:35.026 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.023+0000 7f1365f4aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T15:52:35.026 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.023+0000 7f1365f4aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T15:52:35.026 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.023+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.036+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.036+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.058 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.056+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:35.058 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.056+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.064+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:35.067 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.064+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.081+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.081+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.089+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:35.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.089+0000 7f1365f4aec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T15:52:35.097 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.089+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.098 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.095+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:35.098 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.095+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.105+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.105+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.125+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:35.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.125+0000 7f1365f4aec0 20 zonegroup default 2019-11-29T15:52:35.127 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.125+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.132+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:35.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.132+0000 7f1365f4aec0 10 Cannot find current period zone using local zone 2019-11-29T15:52:35.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.132+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.140+0000 7f1365f4aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.140+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.153 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.151+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:35.153 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.151+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:35.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.166+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:35.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.166+0000 7f1365f4aec0 20 zone default found 2019-11-29T15:52:35.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.166+0000 7f1365f4aec0 20 started sync module instance, tier type = 2019-11-29T15:52:35.169 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.166+0000 7f1365f4aec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T15:52:35.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=0 2019-11-29T15:52:35.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=1 2019-11-29T15:52:35.660 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=2 2019-11-29T15:52:35.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=3 2019-11-29T15:52:35.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=4 2019-11-29T15:52:35.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=5 2019-11-29T15:52:35.661 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.657+0000 7f1365f4aec0 20 add_watcher() i=6 2019-11-29T15:52:35.715 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.712+0000 7f1365f4aec0 20 add_watcher() i=7 2019-11-29T15:52:35.715 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.712+0000 7f1365f4aec0 2 all 8 watchers are set, enabling cache 2019-11-29T15:52:35.715 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.712+0000 7f1323fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T15:52:35.718 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.716+0000 7f1353177700 0 -- 172.21.15.69:0/173280710 >> v1:172.21.15.69:6789/0 conn(0x55da3f018890 legacy=0x55da3f01ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:35.769+0000 7f1353177700 0 -- 172.21.15.69:0/2000826634 >> [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] conn(0x55da3f02d960 msgr2=0x55da3f02fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.272+0000 7f12927fc700 20 reqs_thread_entry: start 2019-11-29T15:52:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.273+0000 7f128bfff700 20 reqs_thread_entry: start 2019-11-29T15:52:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.273+0000 7f1365f4aec0 20 init_complete bucket index max shards: 0 2019-11-29T15:52:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.273+0000 7f128a7fc700 20 reqs_thread_entry: start 2019-11-29T15:52:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.273+0000 7f1365f4aec0 20 get_system_obj_state: rctx=0x7ffd1a147e48 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55da3f097770 s->prefetch_data=0 2019-11-29T15:52:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.273+0000 7f1365f4aec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-29T15:52:37.303 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-11-29T15:52:37.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:37.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T15:52:37.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T15:52:37.304 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T15:52:37.305 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.300+0000 7f1365f4aec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.328+0000 7f1365f4aec0 20 rados_obj.operate() r=0 bl.length=446 2019-11-29T15:52:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.328+0000 7f1365f4aec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-11-29T15:52:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.328+0000 7f1365f4aec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:37.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.526+0000 7f1365f4aec0 10 removing key index: UOMFTAIBECLZKJCOMSGG 2019-11-29T15:52:37.529 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.526+0000 7f1365f4aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-11-29T15:52:37.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.538+0000 7f1338ff9700 10 RGWWatcher::handle_notify() notify_id 584115552840 cookie 94395848215536 notifier 4655 bl.length()=170 2019-11-29T15:52:37.814 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.812+0000 7f1365f4aec0 10 removing email index: bar.client.0+test@test.test 2019-11-29T15:52:37.815 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.812+0000 7f1365f4aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-11-29T15:52:37.827 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:37.825+0000 7f1338ff9700 10 RGWWatcher::handle_notify() notify_id 584115552648 cookie 94395848193200 notifier 4655 bl.length()=178 2019-11-29T15:52:38.258 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:38.255+0000 7f1365f4aec0 10 removing user buckets index 2019-11-29T15:52:38.258 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:38.255+0000 7f1365f4aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-11-29T15:52:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:38.263+0000 7f1338ff9700 10 RGWWatcher::handle_notify() notify_id 584115552841 cookie 94395848196112 notifier 4655 bl.length()=169 2019-11-29T15:52:39.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.136+0000 7f1365f4aec0 10 removing user index: bar.client.0 2019-11-29T15:52:39.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.136+0000 7f1365f4aec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-11-29T15:52:39.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.136+0000 7f1365f4aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-11-29T15:52:39.155 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.152+0000 7f1338ff9700 10 RGWWatcher::handle_notify() notify_id 584115552649 cookie 94395848193200 notifier 4655 bl.length()=161 2019-11-29T15:52:39.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.465+0000 7f1365f4aec0 20 remove_watcher() i=0 2019-11-29T15:52:39.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.465+0000 7f1365f4aec0 2 removed watcher, disabling cache 2019-11-29T15:52:39.497 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.494+0000 7f1365f4aec0 20 remove_watcher() i=1 2019-11-29T15:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.524+0000 7f1365f4aec0 20 remove_watcher() i=2 2019-11-29T15:52:39.575 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.573+0000 7f1365f4aec0 20 remove_watcher() i=3 2019-11-29T15:52:39.609 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.607+0000 7f1365f4aec0 20 remove_watcher() i=4 2019-11-29T15:52:39.646 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.643+0000 7f1365f4aec0 20 remove_watcher() i=5 2019-11-29T15:52:39.696 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.693+0000 7f1365f4aec0 20 remove_watcher() i=6 2019-11-29T15:52:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.743+0000 7f1365f4aec0 20 remove_watcher() i=7 2019-11-29T15:52:39.761 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-11-29T15:52:39.888 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.886+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:39.909 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.907+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:39.909 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.907+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.916+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.916+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:39.923 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.921+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:39.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.921+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:39.932 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:39.930+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:40.155 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.153+0000 7fd222d46ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T15:52:40.155 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.153+0000 7fd222d46ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T15:52:40.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.153+0000 7fd222d46ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T15:52:40.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.153+0000 7fd222d46ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T15:52:40.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.153+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.160+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:40.163 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.160+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.175+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:40.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.175+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.188 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.186+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:40.189 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.186+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.203+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:40.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.203+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.210+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.210+0000 7fd222d46ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T15:52:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.210+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.220+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:40.225 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.220+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.235 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.233+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:40.235 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.233+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.253+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:40.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.253+0000 7fd222d46ec0 20 zonegroup default 2019-11-29T15:52:40.257 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.253+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.261 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.259+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:40.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.259+0000 7fd222d46ec0 10 Cannot find current period zone using local zone 2019-11-29T15:52:40.262 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.259+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.265+0000 7fd222d46ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:40.267 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.265+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.273+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:40.275 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.273+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.286+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.286+0000 7fd222d46ec0 20 zone default found 2019-11-29T15:52:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.286+0000 7fd222d46ec0 20 started sync module instance, tier type = 2019-11-29T15:52:40.289 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.286+0000 7fd222d46ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=0 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=1 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=2 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=3 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=4 2019-11-29T15:52:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=5 2019-11-29T15:52:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.784+0000 7fd222d46ec0 20 add_watcher() i=6 2019-11-29T15:52:40.818 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.816+0000 7fd222d46ec0 20 add_watcher() i=7 2019-11-29T15:52:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.816+0000 7fd222d46ec0 2 all 8 watchers are set, enabling cache 2019-11-29T15:52:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.816+0000 7fd163fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T15:52:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.819+0000 7fd210774700 0 -- 172.21.15.69:0/3447717813 >> v2:172.21.15.69:3300/0 conn(0x55795fafce80 msgr2=0x55795fafd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T15:52:40.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:40.860+0000 7fd210774700 0 -- 172.21.15.69:0/3173265495 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7fd150017570 msgr2=0x7fd150019a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:42.346 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd14e7fc700 20 reqs_thread_entry: start 2019-11-29T15:52:42.346 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd13ffff700 20 reqs_thread_entry: start 2019-11-29T15:52:42.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd222d46ec0 20 init_complete bucket index max shards: 0 2019-11-29T15:52:42.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd13e7fc700 20 reqs_thread_entry: start 2019-11-29T15:52:42.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd222d46ec0 20 get_system_obj_state: rctx=0x7ffc53d991d8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55795fb81ff0 s->prefetch_data=0 2019-11-29T15:52:42.347 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.344+0000 7fd222d46ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-29T15:52:42.373 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-11-29T15:52:42.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:42.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T15:52:42.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T15:52:42.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T15:52:42.374 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.371+0000 7fd222d46ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:42.400 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.398+0000 7fd222d46ec0 20 rados_obj.operate() r=0 bl.length=446 2019-11-29T15:52:42.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.398+0000 7fd222d46ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-11-29T15:52:42.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.398+0000 7fd222d46ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:42.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.520+0000 7fd222d46ec0 10 removing key index: HHBACIXZJFYTJJFSRMUE 2019-11-29T15:52:42.523 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.520+0000 7fd222d46ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-11-29T15:52:42.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:42.539+0000 7fd2057fa700 10 RGWWatcher::handle_notify() notify_id 584115552842 cookie 93979784673264 notifier 5028 bl.length()=170 2019-11-29T15:52:43.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.136+0000 7fd222d46ec0 10 removing email index: foo.client.0+test@test.test 2019-11-29T15:52:43.139 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.136+0000 7fd222d46ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-11-29T15:52:43.156 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.152+0000 7fd2057fa700 10 RGWWatcher::handle_notify() notify_id 584115552424 cookie 93979784709968 notifier 5028 bl.length()=178 2019-11-29T15:52:43.401 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.399+0000 7fd222d46ec0 10 removing user buckets index 2019-11-29T15:52:43.402 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.399+0000 7fd222d46ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-11-29T15:52:43.408 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.405+0000 7fd2057fa700 10 RGWWatcher::handle_notify() notify_id 584115552650 cookie 93979784670032 notifier 5028 bl.length()=169 2019-11-29T15:52:43.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.659+0000 7fd222d46ec0 10 removing user index: foo.client.0 2019-11-29T15:52:43.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.659+0000 7fd222d46ec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-11-29T15:52:43.662 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.659+0000 7fd222d46ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-11-29T15:52:43.685 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:43.683+0000 7fd2057fa700 10 RGWWatcher::handle_notify() notify_id 584115552390 cookie 93979784668064 notifier 5028 bl.length()=161 2019-11-29T15:52:44.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.079+0000 7fd222d46ec0 20 remove_watcher() i=0 2019-11-29T15:52:44.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.079+0000 7fd222d46ec0 2 removed watcher, disabling cache 2019-11-29T15:52:44.124 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.121+0000 7fd222d46ec0 20 remove_watcher() i=1 2019-11-29T15:52:44.165 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.162+0000 7fd222d46ec0 20 remove_watcher() i=2 2019-11-29T15:52:44.205 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.203+0000 7fd222d46ec0 20 remove_watcher() i=3 2019-11-29T15:52:44.260 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.258+0000 7fd222d46ec0 20 remove_watcher() i=4 2019-11-29T15:52:44.320 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.318+0000 7fd222d46ec0 20 remove_watcher() i=5 2019-11-29T15:52:44.388 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.385+0000 7fd222d46ec0 20 remove_watcher() i=6 2019-11-29T15:52:44.428 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.425+0000 7fd222d46ec0 20 remove_watcher() i=7 2019-11-29T15:52:44.438 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-11-29T15:52:44.581 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.578+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.601 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.599+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.602 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.599+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.608+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.608+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.612+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.612+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.622 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.620+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:44.823 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.821+0000 7f9e347f4ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T15:52:44.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.821+0000 7f9e347f4ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T15:52:44.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.821+0000 7f9e347f4ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T15:52:44.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.821+0000 7f9e347f4ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T15:52:44.824 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.821+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.833 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.831+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:44.834 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.831+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.850+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:44.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.850+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.860+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:44.863 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.860+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.879 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.877+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:44.879 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.877+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.884+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.884+0000 7f9e347f4ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T15:52:44.886 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.884+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.891+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.891+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.904+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.904+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.922+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:44.924 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.922+0000 7f9e347f4ec0 20 zonegroup default 2019-11-29T15:52:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.922+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.931+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.931+0000 7f9e347f4ec0 10 Cannot find current period zone using local zone 2019-11-29T15:52:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.931+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.940+0000 7f9e347f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.940+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.949+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:44.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.949+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.965+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.965+0000 7f9e347f4ec0 20 zone default found 2019-11-29T15:52:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.965+0000 7f9e347f4ec0 20 started sync module instance, tier type = 2019-11-29T15:52:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:44.965+0000 7f9e347f4ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T15:52:45.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=0 2019-11-29T15:52:45.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=1 2019-11-29T15:52:45.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=2 2019-11-29T15:52:45.413 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=3 2019-11-29T15:52:45.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=4 2019-11-29T15:52:45.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=5 2019-11-29T15:52:45.414 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.411+0000 7f9e347f4ec0 20 add_watcher() i=6 2019-11-29T15:52:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.445+0000 7f9e347f4ec0 20 add_watcher() i=7 2019-11-29T15:52:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.445+0000 7f9e347f4ec0 2 all 8 watchers are set, enabling cache 2019-11-29T15:52:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.445+0000 7f9df27dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T15:52:45.451 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.450+0000 7f9e21a21700 0 -- 172.21.15.69:0/1351140219 >> v1:172.21.15.69:6789/0 conn(0x555efacc4770 legacy=0x555efacc6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T15:52:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:45.491+0000 7f9e21a21700 0 -- 172.21.15.69:0/2044639656 >> [v2:172.21.15.69:6800/8773,v1:172.21.15.69:6801/8773] conn(0x7f9d640175a0 msgr2=0x7f9d64019a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-29T15:52:46.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.949+0000 7f9d68ff9700 20 reqs_thread_entry: start 2019-11-29T15:52:46.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.950+0000 7f9d627fc700 20 reqs_thread_entry: start 2019-11-29T15:52:46.951 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.950+0000 7f9e347f4ec0 20 init_complete bucket index max shards: 0 2019-11-29T15:52:46.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.950+0000 7f9d60ff9700 20 reqs_thread_entry: start 2019-11-29T15:52:46.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.950+0000 7f9e347f4ec0 20 get_system_obj_state: rctx=0x7ffc96482d48 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x555efad47640 s->prefetch_data=0 2019-11-29T15:52:46.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.950+0000 7f9e347f4ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-29T15:52:46.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-11-29T15:52:46.962 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T15:52:46.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-29T15:52:46.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-29T15:52:46.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-29T15:52:46.963 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.961+0000 7f9e347f4ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.977+0000 7f9e347f4ec0 20 rados_obj.operate() r=0 bl.length=505 2019-11-29T15:52:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.977+0000 7f9e347f4ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-11-29T15:52:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.977+0000 7f9e347f4ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-29T15:52:46.989 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.987+0000 7f9e347f4ec0 10 removing key index: MQHLCBQGRZVGECKEKFEQ 2019-11-29T15:52:46.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.987+0000 7f9e347f4ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=170 2019-11-29T15:52:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:46.995+0000 7f9e02ffd700 10 RGWWatcher::handle_notify() notify_id 584115552651 cookie 93866422873728 notifier 5046 bl.length()=170 2019-11-29T15:52:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.453+0000 7f9e347f4ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-11-29T15:52:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.453+0000 7f9e347f4ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-11-29T15:52:47.469 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.468+0000 7f9e02ffd700 10 RGWWatcher::handle_notify() notify_id 584115552425 cookie 93866422901056 notifier 5046 bl.length()=193 2019-11-29T15:52:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.755+0000 7f9e347f4ec0 10 removing user buckets index 2019-11-29T15:52:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.755+0000 7f9e347f4ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-11-29T15:52:47.773 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.772+0000 7f9e02ffd700 10 RGWWatcher::handle_notify() notify_id 584115552426 cookie 93866422914128 notifier 5046 bl.length()=184 2019-11-29T15:52:47.972 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.971+0000 7f9e347f4ec0 10 removing user index: testx$tenanteduser.client.0 2019-11-29T15:52:47.972 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.971+0000 7f9e347f4ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-11-29T15:52:47.973 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.971+0000 7f9e347f4ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-11-29T15:52:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:47.980+0000 7f9e02ffd700 10 RGWWatcher::handle_notify() notify_id 584115552843 cookie 93866422897520 notifier 5046 bl.length()=176 2019-11-29T15:52:48.620 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.618+0000 7f9e347f4ec0 20 remove_watcher() i=0 2019-11-29T15:52:48.620 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.618+0000 7f9e347f4ec0 2 removed watcher, disabling cache 2019-11-29T15:52:48.668 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.666+0000 7f9e347f4ec0 20 remove_watcher() i=1 2019-11-29T15:52:48.709 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.708+0000 7f9e347f4ec0 20 remove_watcher() i=2 2019-11-29T15:52:48.755 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.754+0000 7f9e347f4ec0 20 remove_watcher() i=3 2019-11-29T15:52:48.946 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:48.945+0000 7f9e347f4ec0 20 remove_watcher() i=4 2019-11-29T15:52:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:49.188+0000 7f9e347f4ec0 20 remove_watcher() i=5 2019-11-29T15:52:49.243 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:49.241+0000 7f9e347f4ec0 20 remove_watcher() i=6 2019-11-29T15:52:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:49.306+0000 7f9e347f4ec0 20 remove_watcher() i=7 2019-11-29T15:52:49.316 INFO:tasks.s3tests:Removing s3-tests... 2019-11-29T15:52:49.316 INFO:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-11-29T15:52:49.464 INFO:teuthology.run_tasks:Running task swift... 2019-11-29T15:52:49.505 INFO:tasks.swift:clients=['client.0'] 2019-11-29T15:52:49.505 INFO:tasks.swift:Downloading swift... 2019-11-29T15:52:49.506 INFO:teuthology.orchestra.run.smithi069:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-11-29T15:52:49.603 INFO:teuthology.orchestra.run.smithi069.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-11-29T15:52:50.060 INFO:tasks.swift:Creating rgw users... 2019-11-29T15:52:50.060 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret JHdLMU3WKx3+bhf4d6Zk1r2z1/v9YdJ+JL2Bzt5mnXwggQP49uWjvA== --email foo.client.0+test@test.test --key-type swift --access full 2019-11-29T15:52:50.239 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.238+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.258 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.256+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.258 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.256+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.265+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.266 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.265+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.271+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.271+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.278+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:50.434 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.433+0000 7fbe8a1d1ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T15:52:50.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.433+0000 7fbe8a1d1ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T15:52:50.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.433+0000 7fbe8a1d1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T15:52:50.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.433+0000 7fbe8a1d1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T15:52:50.435 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.433+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.443+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.443+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.465+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:50.467 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.465+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.474+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:50.476 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.474+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.491+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.491+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.504 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.503+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.503+0000 7fbe8a1d1ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T15:52:50.505 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.503+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.511 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.509+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.512 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.509+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.527+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.527+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.555+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.555+0000 7fbe8a1d1ec0 20 zonegroup default 2019-11-29T15:52:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.555+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.563+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.564 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.563+0000 7fbe8a1d1ec0 10 Cannot find current period zone using local zone 2019-11-29T15:52:50.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.563+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.571 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.570+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:50.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.570+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.577+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:50.579 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.577+0000 7fbe8a1d1ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.591+0000 7fbe8a1d1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.591+0000 7fbe8a1d1ec0 20 zone default found 2019-11-29T15:52:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.591+0000 7fbe8a1d1ec0 20 started sync module instance, tier type = 2019-11-29T15:52:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.591+0000 7fbe8a1d1ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T15:52:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=0 2019-11-29T15:52:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=1 2019-11-29T15:52:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=2 2019-11-29T15:52:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=3 2019-11-29T15:52:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=4 2019-11-29T15:52:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.952+0000 7fbe8a1d1ec0 20 add_watcher() i=5 2019-11-29T15:52:50.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.970+0000 7fbe8a1d1ec0 20 add_watcher() i=6 2019-11-29T15:52:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.988+0000 7fbe8a1d1ec0 20 add_watcher() i=7 2019-11-29T15:52:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.988+0000 7fbe8a1d1ec0 2 all 8 watchers are set, enabling cache 2019-11-29T15:52:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.989+0000 7fbe47fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T15:52:50.993 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:50.991+0000 7fbe6ffff700 0 -- 172.21.15.69:0/1511571359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55b2cc04ca80 msgr2=0x55b2cc04ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-29T15:52:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:51.026+0000 7fbe77bff700 0 -- 172.21.15.69:0/342533800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2cc049720 msgr2=0x55b2cc049b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:52.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbdb67fc700 20 reqs_thread_entry: start 2019-11-29T15:52:52.480 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbdaffff700 20 reqs_thread_entry: start 2019-11-29T15:52:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbe8a1d1ec0 20 init_complete bucket index max shards: 0 2019-11-29T15:52:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbdae7fc700 20 reqs_thread_entry: start 2019-11-29T15:52:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbe8a1d1ec0 20 get_system_obj_state: rctx=0x7ffe2e5c3db8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55b2cc0c1ab0 s->prefetch_data=0 2019-11-29T15:52:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.479+0000 7fbe8a1d1ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-29T15:52:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.487+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-11-29T15:52:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.487+0000 7fbe8a1d1ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.487+0000 7fbe8a1d1ec0 20 get_system_obj_state: rctx=0x7ffe2e5c3dc8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x55b2cc0c3840 s->prefetch_data=0 2019-11-29T15:52:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.487+0000 7fbe8a1d1ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-11-29T15:52:52.494 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.493+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-11-29T15:52:52.495 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.493+0000 7fbe8a1d1ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T15:52:52.495 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.493+0000 7fbe8a1d1ec0 20 get_system_obj_state: rctx=0x7ffe2e5c39d8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55b2cc0c3840 s->prefetch_data=0 2019-11-29T15:52:52.495 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.493+0000 7fbe8a1d1ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-11-29T15:52:52.501 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.500+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-11-29T15:52:52.501 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.500+0000 7fbe8a1d1ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-29T15:52:52.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.500+0000 7fbe8a1d1ec0 20 get_system_obj_state: rctx=0x7ffe2e5c3b58 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55b2cc0c3af0 s->prefetch_data=0 2019-11-29T15:52:52.502 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.500+0000 7fbe8a1d1ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-11-29T15:52:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.506+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-11-29T15:52:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.506+0000 7fbe8a1d1ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-29T15:52:52.542 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.540+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T15:52:52.542 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.540+0000 7fbe8a1d1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:52.542 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.540+0000 7fbe8a1d1ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-11-29T15:52:52.552 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.550+0000 7fbe6cff9700 10 RGWWatcher::handle_notify() notify_id 584115552391 cookie 94226415278048 notifier 4679 bl.length()=612 2019-11-29T15:52:52.552 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.551+0000 7fbe6cff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T15:52:52.552 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:52.551+0000 7fbe6cff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:53.280 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.279+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-29T15:52:53.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.279+0000 7fbe8a1d1ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T15:52:53.281 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.279+0000 7fbe8a1d1ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-11-29T15:52:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.298+0000 7fbe6cff9700 10 RGWWatcher::handle_notify() notify_id 584115552427 cookie 94226415306224 notifier 4679 bl.length()=194 2019-11-29T15:52:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.298+0000 7fbe6cff9700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-29T15:52:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:53.298+0000 7fbe6cff9700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-29T15:52:54.024 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.022+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-11-29T15:52:54.024 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.022+0000 7fbe8a1d1ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-29T15:52:54.024 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.022+0000 7fbe8a1d1ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-11-29T15:52:54.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.033+0000 7fbe6cff9700 10 RGWWatcher::handle_notify() notify_id 584115552652 cookie 94226415281488 notifier 4679 bl.length()=183 2019-11-29T15:52:54.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.033+0000 7fbe6cff9700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-11-29T15:52:54.034 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.033+0000 7fbe6cff9700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-29T15:52:54.425 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.424+0000 7fbe8a1d1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T15:52:54.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.424+0000 7fbe8a1d1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:54.426 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.424+0000 7fbe8a1d1ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-11-29T15:52:54.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.440+0000 7fbe6cff9700 10 RGWWatcher::handle_notify() notify_id 584115552392 cookie 94226415278048 notifier 4679 bl.length()=636 2019-11-29T15:52:54.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.440+0000 7fbe6cff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-29T15:52:54.442 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.440+0000 7fbe6cff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-29T15:52:54.539 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T15:52:54.539 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "foo.client.0", 2019-11-29T15:52:54.539 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. foo.client.0 foo", 2019-11-29T15:52:54.539 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "foo.client.0+test@test.test", 2019-11-29T15:52:54.539 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [ 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: "id": "foo.client.0:foo", 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: "permissions": "full-control" 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T15:52:54.540 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [], 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [ 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "foo.client.0:foo", 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "JHdLMU3WKx3+bhf4d6Zk1r2z1/v9YdJ+JL2Bzt5mnXwggQP49uWjvA==" 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T15:52:54.541 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T15:52:54.542 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T15:52:54.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T15:52:54.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T15:52:54.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T15:52:54.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T15:52:54.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T15:52:54.545 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T15:52:54.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T15:52:54.545 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T15:52:54.545 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T15:52:54.545 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T15:52:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.606+0000 7fbe8a1d1ec0 20 remove_watcher() i=0 2019-11-29T15:52:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.606+0000 7fbe8a1d1ec0 2 removed watcher, disabling cache 2019-11-29T15:52:54.643 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.638+0000 7fbe8a1d1ec0 20 remove_watcher() i=1 2019-11-29T15:52:54.669 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.668+0000 7fbe8a1d1ec0 20 remove_watcher() i=2 2019-11-29T15:52:54.710 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.708+0000 7fbe8a1d1ec0 20 remove_watcher() i=3 2019-11-29T15:52:54.751 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.750+0000 7fbe8a1d1ec0 20 remove_watcher() i=4 2019-11-29T15:52:54.799 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.798+0000 7fbe8a1d1ec0 20 remove_watcher() i=5 2019-11-29T15:52:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.836+0000 7fbe8a1d1ec0 20 remove_watcher() i=6 2019-11-29T15:52:54.884 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:54.883+0000 7fbe8a1d1ec0 20 remove_watcher() i=7 2019-11-29T15:52:54.897 INFO:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret a9G/TVzljc1Pti1nCxP3SNEJt/InkdjWUAw9Ogt+Qu5iA0l+aQUYBg== --email bar.client.0+test@test.test --key-type swift --access full 2019-11-29T15:52:55.081 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.079+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.116 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.115+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.117 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.115+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.130 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.129+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.131 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.129+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.133+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.135 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.133+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.143 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.142+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:55.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.331+0000 7f6717868ec0 20 RGWRados::pool_iterate: got zonegroup_info.342f3735-2c13-49e6-8da5-82e456262401 2019-11-29T15:52:55.333 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.331+0000 7f6717868ec0 20 RGWRados::pool_iterate: got zone_info.42bb32be-1e05-491f-815b-a606bdf24d77 2019-11-29T15:52:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.331+0000 7f6717868ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-29T15:52:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.331+0000 7f6717868ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-29T15:52:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.331+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.340+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.340+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.358+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.358+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.370 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.369+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:55.371 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.369+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.387 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.386+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.386+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.393+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.393+0000 7f6717868ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-29T15:52:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.393+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.401+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.401+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.415+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:55.416 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.415+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.431+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-29T15:52:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.431+0000 7f6717868ec0 20 zonegroup default 2019-11-29T15:52:55.434 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.431+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.439+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.439+0000 7f6717868ec0 10 Cannot find current period zone using local zone 2019-11-29T15:52:55.441 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.439+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.448 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.447+0000 7f6717868ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-29T15:52:55.448 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.447+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.457 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.455+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-29T15:52:55.458 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.456+0000 7f6717868ec0 20 rados->read ofs=0 len=0 2019-11-29T15:52:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.475+0000 7f6717868ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-29T15:52:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.475+0000 7f6717868ec0 20 zone default found 2019-11-29T15:52:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.475+0000 7f6717868ec0 20 started sync module instance, tier type = 2019-11-29T15:52:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.475+0000 7f6717868ec0 20 started zone id=42bb32be-1e05-491f-815b-a606bdf24d77 (name=default) with tier type = 2019-11-29T15:52:55.914 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=0 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=1 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=2 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=3 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=4 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=5 2019-11-29T15:52:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.913+0000 7f6717868ec0 20 add_watcher() i=6 2019-11-29T15:52:55.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.970+0000 7f6717868ec0 20 add_watcher() i=7 2019-11-29T15:52:55.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.970+0000 7f6717868ec0 2 all 8 watchers are set, enabling cache 2019-11-29T15:52:55.971 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.970+0000 7f66d57e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-29T15:52:55.987 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:55.985+0000 7f6705296700 0 -- 172.21.15.69:0/1208053778 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558d03942490 msgr2=0x558d039428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:56.017 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:56.015+0000 7f6705296700 0 -- 172.21.15.69:0/2481393385 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558d03948560 msgr2=0x558d03948970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-29T15:52:57.540 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.538+0000 7f66477fe700 20 reqs_thread_entry: start 2019-11-29T15:52:57.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.538+0000 7f66457fa700 20 reqs_thread_entry: start 2019-11-29T15:52:57.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.539+0000 7f6717868ec0 20 init_complete bucket index max shards: 0 2019-11-29T15:52:57.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.539+0000 7f66337fe700 20 reqs_thread_entry: start 2019-11-29T15:52:57.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.539+0000 7f6717868ec0 20 get_system_obj_state: rctx=0x7ffe60aef098 obj=default.rgw.meta:users.uid:bar.client.0 state=0x558d039c49c0 s->prefetch_data=0 2019-11-29T15:52:57.541 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.539+0000 7f6717868ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-29T15:52:57.555 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.552+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-11-29T15:52:57.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.552+0000 7f6717868ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:57.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.552+0000 7f6717868ec0 20 get_system_obj_state: rctx=0x7ffe60aef0a8 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x558d039c6750 s->prefetch_data=0 2019-11-29T15:52:57.556 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.552+0000 7f6717868ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-11-29T15:52:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.559+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-11-29T15:52:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.559+0000 7f6717868ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T15:52:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.559+0000 7f6717868ec0 20 get_system_obj_state: rctx=0x7ffe60aeecb8 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x558d039c6750 s->prefetch_data=0 2019-11-29T15:52:57.565 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.559+0000 7f6717868ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-11-29T15:52:57.568 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.566+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-11-29T15:52:57.569 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.566+0000 7f6717868ec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-29T15:52:57.569 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.566+0000 7f6717868ec0 20 get_system_obj_state: rctx=0x7ffe60aeee38 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x558d039c6a00 s->prefetch_data=0 2019-11-29T15:52:57.569 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.566+0000 7f6717868ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-11-29T15:52:57.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.571+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-11-29T15:52:57.572 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.571+0000 7f6717868ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-29T15:52:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.637+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T15:52:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.637+0000 7f6717868ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.637+0000 7f6717868ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-11-29T15:52:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.649+0000 7f66fa290700 10 RGWWatcher::handle_notify() notify_id 584115552653 cookie 94064138698880 notifier 4981 bl.length()=612 2019-11-29T15:52:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.649+0000 7f66fa290700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T15:52:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:57.649+0000 7f66fa290700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.146+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-29T15:52:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.146+0000 7f6717868ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T15:52:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.146+0000 7f6717868ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-11-29T15:52:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.157+0000 7f66fa290700 10 RGWWatcher::handle_notify() notify_id 584115552654 cookie 94064138698880 notifier 4981 bl.length()=194 2019-11-29T15:52:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.157+0000 7f66fa290700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-29T15:52:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.157+0000 7f66fa290700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-29T15:52:58.597 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.596+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-11-29T15:52:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.596+0000 7f6717868ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-29T15:52:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.596+0000 7f6717868ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-11-29T15:52:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.615+0000 7f66fa290700 10 RGWWatcher::handle_notify() notify_id 584115552844 cookie 94064138722704 notifier 4981 bl.length()=183 2019-11-29T15:52:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.615+0000 7f66fa290700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-11-29T15:52:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:58.615+0000 7f66fa290700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-29T15:52:59.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.161+0000 7f6717868ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T15:52:59.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.161+0000 7f6717868ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:59.162 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.161+0000 7f6717868ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-11-29T15:52:59.178 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.177+0000 7f66fa290700 10 RGWWatcher::handle_notify() notify_id 584115552655 cookie 94064138698880 notifier 4981 bl.length()=636 2019-11-29T15:52:59.179 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.177+0000 7f66fa290700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-29T15:52:59.179 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.177+0000 7f66fa290700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-29T15:52:59.781 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "user_id": "bar.client.0", 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "display_name": "Mr. bar.client.0 bar", 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "email": "bar.client.0+test@test.test", 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "suspended": 0, 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max_buckets": 1000, 2019-11-29T15:52:59.782 INFO:teuthology.orchestra.run.smithi069.stdout: "subusers": [ 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: "id": "bar.client.0:bar", 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: "permissions": "full-control" 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: "keys": [], 2019-11-29T15:52:59.783 INFO:teuthology.orchestra.run.smithi069.stdout: "swift_keys": [ 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: { 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "user": "bar.client.0:bar", 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "secret_key": "a9G/TVzljc1Pti1nCxP3SNEJt/InkdjWUAw9Ogt+Qu5iA0l+aQUYBg==" 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: } 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": [], 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "op_mask": "read, write, delete", 2019-11-29T15:52:59.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_placement": "", 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_storage_class": "", 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "placement_tags": [], 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "bucket_quota": { 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T15:52:59.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "user_quota": { 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": false, 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "check_on_raw": false, 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size": -1, 2019-11-29T15:52:59.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max_size_kb": 0, 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max_objects": -1 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: "temp_url_keys": [], 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "rgw", 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: "mfa_ids": [] 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout:} 2019-11-29T15:52:59.787 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T15:52:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.850+0000 7f6717868ec0 20 remove_watcher() i=0 2019-11-29T15:52:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.850+0000 7f6717868ec0 2 removed watcher, disabling cache 2019-11-29T15:52:59.905 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.904+0000 7f6717868ec0 20 remove_watcher() i=1 2019-11-29T15:52:59.952 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:52:59.950+0000 7f6717868ec0 20 remove_watcher() i=2 2019-11-29T15:53:00.012 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:53:00.011+0000 7f6717868ec0 20 remove_watcher() i=3 2019-11-29T15:53:00.053 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:53:00.052+0000 7f6717868ec0 20 remove_watcher() i=4 2019-11-29T15:53:00.095 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:53:00.093+0000 7f6717868ec0 20 remove_watcher() i=5 2019-11-29T15:53:00.160 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:53:00.159+0000 7f6717868ec0 20 remove_watcher() i=6 2019-11-29T15:53:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2019-11-29T15:53:00.204+0000 7f6717868ec0 20 remove_watcher() i=7 2019-11-29T15:53:00.219 INFO:tasks.swift:Configuring testswift... 2019-11-29T15:53:00.219 INFO:teuthology.orchestra.run.smithi069:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-11-29T15:53:02.899 INFO:teuthology.orchestra.run.smithi069.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-11-29T15:53:04.258 INFO:teuthology.orchestra.run.smithi069.stdout:Installing setuptools, pip, wheel...done. 2019-11-29T15:53:04.523 INFO:teuthology.orchestra.run.smithi069.stderr:Cache entry deserialization failed, entry ignored 2019-11-29T15:53:04.646 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting pip 2019-11-29T15:53:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-11-29T15:53:04.790 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: pip 2019-11-29T15:53:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: pip 9.0.1 2019-11-29T15:53:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling pip-9.0.1: 2019-11-29T15:53:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled pip-9.0.1 2019-11-29T15:53:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed pip-19.3.1 2019-11-29T15:53:05.927 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T15:53:06.384 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting setuptools==32.3.1 2019-11-29T15:53:06.387 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-11-29T15:53:06.426 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: setuptools 2019-11-29T15:53:06.427 INFO:teuthology.orchestra.run.smithi069.stdout: Found existing installation: setuptools 28.8.0 2019-11-29T15:53:06.441 INFO:teuthology.orchestra.run.smithi069.stdout: Uninstalling setuptools-28.8.0: 2019-11-29T15:53:06.449 INFO:teuthology.orchestra.run.smithi069.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-29T15:53:06.581 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed setuptools-32.3.1 2019-11-29T15:53:06.984 INFO:teuthology.orchestra.run.smithi069.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-29T15:53:07.182 INFO:teuthology.orchestra.run.smithi069.stdout:Processing /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl 2019-11-29T15:53:07.230 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting nose>=1.0.0 2019-11-29T15:53:07.231 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-11-29T15:53:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting boto>=2.0b4 2019-11-29T15:53:07.354 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-11-29T15:53:07.490 INFO:teuthology.orchestra.run.smithi069.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-11-29T15:53:07.733 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting gevent>=0.14 2019-11-29T15:53:07.741 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T15:53:07.979 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting isodate>=0.4.4 2019-11-29T15:53:07.980 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-11-29T15:53:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting eventlet>=0.9.14 2019-11-29T15:53:08.152 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-11-29T15:53:08.316 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting netifaces>=0.5 2019-11-29T15:53:08.326 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T15:53:08.412 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting xattr<0.7.0,>=0.6 2019-11-29T15:53:08.422 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-11-29T15:53:08.823 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting simplejson>=2.1.3 2019-11-29T15:53:08.833 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-11-29T15:53:09.276 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting WebOb==1.0 2019-11-29T15:53:09.288 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-11-29T15:53:09.739 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-29T15:53:09.741 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-29T15:53:09.800 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting six 2019-11-29T15:53:09.802 INFO:teuthology.orchestra.run.smithi069.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-11-29T15:53:09.852 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting monotonic>=1.4 2019-11-29T15:53:09.862 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-11-29T15:53:09.928 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting dnspython>=1.15.0 2019-11-29T15:53:09.939 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-11-29T15:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout:Collecting enum34; python_version < "3.4" 2019-11-29T15:53:10.081 INFO:teuthology.orchestra.run.smithi069.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-11-29T15:53:10.111 INFO:teuthology.orchestra.run.smithi069.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-11-29T15:53:10.112 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for xattr (setup.py): started 2019-11-29T15:53:10.722 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-11-29T15:53:10.723 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24122 sha256=bf89526f944ad1f07543d9827031d99bb9b138074f7ffb314fe5c563e8cc28cd 2019-11-29T15:53:10.723 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-11-29T15:53:10.729 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for simplejson (setup.py): started 2019-11-29T15:53:11.698 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-11-29T15:53:11.699 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113121 sha256=e6d70035b4ddfaf1d17d7fdec00eb0925bd4b32bdf184dad473cc459d9ddd4c4 2019-11-29T15:53:11.699 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-11-29T15:53:11.719 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for WebOb (setup.py): started 2019-11-29T15:53:12.035 INFO:teuthology.orchestra.run.smithi069.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-11-29T15:53:12.036 INFO:teuthology.orchestra.run.smithi069.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=c39abd44d15221281c64af6472bd744aafc10c98fff14168cda8f122436bd981 2019-11-29T15:53:12.036 INFO:teuthology.orchestra.run.smithi069.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-11-29T15:53:12.043 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully built xattr simplejson WebOb 2019-11-29T15:53:12.089 INFO:teuthology.orchestra.run.smithi069.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-11-29T15:53:13.473 INFO:teuthology.orchestra.run.smithi069.stdout:Successfully installed PyYAML-5.1.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-11-29T15:53:13.724 INFO:teuthology.orchestra.run.smithi069.stdout:running develop 2019-11-29T15:53:13.768 INFO:teuthology.orchestra.run.smithi069.stdout:running egg_info 2019-11-29T15:53:13.768 INFO:teuthology.orchestra.run.smithi069.stdout:creating swift.egg-info 2019-11-29T15:53:13.768 INFO:teuthology.orchestra.run.smithi069.stdout:writing swift.egg-info/PKG-INFO 2019-11-29T15:53:13.768 INFO:teuthology.orchestra.run.smithi069.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-11-29T15:53:13.769 INFO:teuthology.orchestra.run.smithi069.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-11-29T15:53:13.772 INFO:teuthology.orchestra.run.smithi069.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-11-29T15:53:13.772 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-11-29T15:53:13.776 INFO:teuthology.orchestra.run.smithi069.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-11-29T15:53:13.776 INFO:teuthology.orchestra.run.smithi069.stdout:reading manifest template 'MANIFEST.in' 2019-11-29T15:53:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:warning: no files found matching 'ChangeLog' 2019-11-29T15:53:13.780 INFO:teuthology.orchestra.run.smithi069.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-11-29T15:53:13.781 INFO:teuthology.orchestra.run.smithi069.stdout:running build_ext 2019-11-29T15:53:13.781 INFO:teuthology.orchestra.run.smithi069.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-11-29T15:53:13.781 INFO:teuthology.orchestra.run.smithi069.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-11-29T15:53:13.797 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.797 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.800 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.801 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.801 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.802 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.803 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.804 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.805 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.805 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.806 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.808 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.809 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.811 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.811 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.812 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.813 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.814 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.815 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.816 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.819 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.821 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.828 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.829 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.831 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.832 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.835 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.836 INFO:teuthology.orchestra.run.smithi069.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-29T15:53:13.836 INFO:teuthology.orchestra.run.smithi069.stdout: 2019-11-29T15:53:13.836 INFO:teuthology.orchestra.run.smithi069.stdout:Installed /home/ubuntu/cephtest/swift 2019-11-29T15:53:13.837 INFO:teuthology.orchestra.run.smithi069.stdout:Processing dependencies for swift==1.4.4 2019-11-29T15:53:13.837 INFO:teuthology.orchestra.run.smithi069.stdout:Finished processing dependencies for swift==1.4.4 2019-11-29T15:53:13.849 INFO:teuthology.orchestra.run.smithi069:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-11-29T15:53:13.906 INFO:teuthology.orchestra.run.smithi069:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-11-29T15:53:48.248 INFO:teuthology.orchestra.run.smithi069.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-11-29T15:54:01.643 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:41:14.533 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.650 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:41:14.540 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.670 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:41:14.445 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.671 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:41:14.560 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.674 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:41:14.450 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.674 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:41:14.564 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.842 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:41:14.618 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.845 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:41:14.734 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:01.870 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:41:14.760 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:54:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-11-29T15:58:18.992 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:45:31.883 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.010 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:45:31.900 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.018 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:45:31.908 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.039 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:45:31.929 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.052 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:45:31.827 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.054 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:45:31.830 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.122 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:45:32.012 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.167 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:45:31.942 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:19.217 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:45:32.104 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:58:38.581 INFO:teuthology.orchestra.run.smithi069.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-11-29T15:58:56.136 INFO:teuthology.orchestra.run.smithi069.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-11-29T15:58:57.290 INFO:teuthology.orchestra.run.smithi069.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-11-29T15:59:13.240 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:46:26.131 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.248 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:46:26.138 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.259 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:46:26.035 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.260 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:46:26.150 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.263 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:46:26.153 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.264 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:46:26.040 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.298 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:46:26.188 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.316 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:46:26.205 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:13.359 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:46:26.135 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T15:59:15.184 INFO:teuthology.orchestra.run.smithi069.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-11-29T15:59:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-11-29T16:00:48.560 INFO:teuthology.orchestra.run.smithi069.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-11-29T16:00:49.142 INFO:teuthology.orchestra.run.smithi069.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-11-29T16:04:24.285 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:51:37.172 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.288 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:51:37.178 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.297 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:51:37.186 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.303 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:51:37.191 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.321 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:51:37.093 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.344 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:51:37.120 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.514 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:51:37.402 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.518 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:51:37.408 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:04:24.624 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:51:37.400 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.529 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:53:53.418 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.535 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:53:53.425 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.559 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:53:53.448 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.582 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:53:53.471 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.609 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:53:53.384 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.611 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:53:53.387 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.638 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:53:53.528 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.663 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:53:53.553 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:06:40.676 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:53:53.452 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:07:29.121 INFO:teuthology.orchestra.run.smithi069.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-11-29T16:07:31.501 INFO:teuthology.orchestra.run.smithi069.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-11-29T16:08:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:08:25.078 INFO:teuthology.orchestra.run.smithi069.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:09:34.285 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:56:47.175 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.290 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:56:47.180 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.301 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:56:47.190 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.305 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:56:47.195 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.306 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:56:47.081 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.313 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:56:47.089 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.419 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:04:56:47.308 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.455 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:04:56:47.346 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:09:34.478 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:04:56:47.254 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:13:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:13:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:13:50.148 INFO:teuthology.orchestra.run.smithi069.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:14:10.375 INFO:teuthology.orchestra.run.smithi069.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:14:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:14:46.219 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:01:59.109 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.231 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:01:59.121 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.239 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:01:59.129 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.245 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:01:59.134 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.247 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:01:59.022 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.254 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:01:59.029 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.414 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:01:59.305 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.470 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:01:59.357 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:14:46.485 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:01:59.260 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:01.955 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:02:14.845 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:01.960 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:02:14.851 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.009 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:02:14.785 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.013 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:02:14.787 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.028 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:02:14.918 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.045 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:02:14.821 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.064 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:02:14.953 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.067 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:02:14.957 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:15:02.088 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:02:14.977 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:16:56.169 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:16:56.157+0000 1694c700 -1 osd.2 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:16.004228+0000 front 2019-11-29T16:16:16.730429+0000 (oldest deadline 2019-11-29T16:16:55.690731+0000) 2019-11-29T16:16:56.699 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:16:56.673+0000 1694c700 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:11.045683+0000 front 2019-11-29T16:16:09.246154+0000 (oldest deadline 2019-11-29T16:16:56.152297+0000) 2019-11-29T16:16:57.136 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:16:57.131+0000 1694c700 -1 osd.2 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:16.004228+0000 front 2019-11-29T16:16:16.730429+0000 (oldest deadline 2019-11-29T16:16:55.690731+0000) 2019-11-29T16:16:57.674 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:16:57.669+0000 1694c700 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:11.045683+0000 front 2019-11-29T16:16:09.246154+0000 (oldest deadline 2019-11-29T16:16:56.152297+0000) 2019-11-29T16:16:57.879 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:16:57.866+0000 1694c700 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:11.340078+0000 front 2019-11-29T16:16:11.374570+0000 (oldest deadline 2019-11-29T16:16:57.239068+0000) 2019-11-29T16:16:58.156 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:16:58.144+0000 1694c700 -1 osd.2 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:16.004228+0000 front 2019-11-29T16:16:16.730429+0000 (oldest deadline 2019-11-29T16:16:55.690731+0000) 2019-11-29T16:16:58.187 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:16:58.178+0000 1694c700 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:14.019038+0000 front 2019-11-29T16:16:16.005539+0000 (oldest deadline 2019-11-29T16:16:57.478481+0000) 2019-11-29T16:16:58.924 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:16:58.921+0000 1694c700 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:16:11.340078+0000 front 2019-11-29T16:16:11.374570+0000 (oldest deadline 2019-11-29T16:16:57.239068+0000) 2019-11-29T16:17:25.705 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:25.671+0000 34271700 -1 osd.0 142 build_incremental_map_msg missing incremental map 141 2019-11-29T16:17:27.627 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:04:40.518 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.662 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:04:40.552 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.724 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:04:40.614 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.726 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:04:40.616 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.737 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:04:40.513 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.740 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:04:40.516 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.854 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:04:40.741 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.883 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:04:40.772 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:27.995 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:04:40.771 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:17:48.173 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:17:47.961+0000 20e22700 -1 RGWWatcher::handle_error cookie 821003888 err (107) Transport endpoint is not connected 2019-11-29T16:17:52.559 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:52.552+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:53.517 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:53.514+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:54.519 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:54.516+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:55.557 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:55.554+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:56.575 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:56.572+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:56.580 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:17:56.568+0000 1694c700 -1 osd.4 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.507709+0000 front 2019-11-29T16:17:12.524784+0000 (oldest deadline 2019-11-29T16:17:56.333613+0000) 2019-11-29T16:17:56.623 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:17:56.616+0000 1694c700 -1 osd.2 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:15.126494+0000 front 2019-11-29T16:17:15.409913+0000 (oldest deadline 2019-11-29T16:17:56.077601+0000) 2019-11-29T16:17:56.726 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:17:56.711+0000 1694c700 -1 osd.7 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.526635+0000 front 2019-11-29T16:17:12.389049+0000 (oldest deadline 2019-11-29T16:17:56.025872+0000) 2019-11-29T16:17:57.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:17:57.576+0000 1694c700 -1 osd.4 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.507709+0000 front 2019-11-29T16:17:12.524784+0000 (oldest deadline 2019-11-29T16:17:56.333613+0000) 2019-11-29T16:17:57.600 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:17:57.597+0000 1694c700 -1 osd.2 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:15.126494+0000 front 2019-11-29T16:17:15.409913+0000 (oldest deadline 2019-11-29T16:17:56.077601+0000) 2019-11-29T16:17:57.615 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:57.611+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:57.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:17:57.747+0000 1694c700 -1 osd.7 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.526635+0000 front 2019-11-29T16:17:12.389049+0000 (oldest deadline 2019-11-29T16:17:56.025872+0000) 2019-11-29T16:17:58.176 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:17:58.168+0000 1694c700 -1 osd.6 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:13.331048+0000 front 2019-11-29T16:17:13.329968+0000 (oldest deadline 2019-11-29T16:17:58.137531+0000) 2019-11-29T16:17:58.548 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:17:58.542+0000 1694c700 -1 osd.4 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.507709+0000 front 2019-11-29T16:17:12.524784+0000 (oldest deadline 2019-11-29T16:17:56.333613+0000) 2019-11-29T16:17:58.557 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:17:58.553+0000 1694c700 -1 osd.2 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:15.126494+0000 front 2019-11-29T16:17:15.409913+0000 (oldest deadline 2019-11-29T16:17:56.077601+0000) 2019-11-29T16:17:58.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:58.641+0000 1694c700 -1 osd.0 142 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:58.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:17:58.752+0000 1694c700 -1 osd.7 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:12.526635+0000 front 2019-11-29T16:17:12.389049+0000 (oldest deadline 2019-11-29T16:17:56.025872+0000) 2019-11-29T16:17:58.780 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:17:58.776+0000 1694c700 -1 osd.5 142 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:17:13.471548+0000 front 2019-11-29T16:17:13.483586+0000 (oldest deadline 2019-11-29T16:17:58.005628+0000) 2019-11-29T16:17:59.828 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:59.812+0000 1694c700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e140) 2019-11-29T16:17:59.831 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:17:59.820+0000 34271700 -1 osd.0 143 build_incremental_map_msg missing incremental map 141 2019-11-29T16:19:01.713 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:19:01.694+0000 1694c700 -1 osd.1 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:96245 6.1 6.f221d291 (undecoded) ondisk+read+known_if_redirected e142) 2019-11-29T16:19:10.384 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:06:23.273 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.393 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:06:23.279 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.398 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:06:23.173 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.399 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:06:23.288 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.402 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:06:23.178 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.407 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:06:23.294 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.521 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:06:23.296 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.528 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:06:23.417 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:10.540 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:06:23.430 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:19:36.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:19:36.197+0000 1694c700 -1 osd.0 147 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected e145) 2019-11-29T16:19:38.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:19:38.167+0000 1694c700 -1 osd.1 147 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:96482 6.1 6:894b844f:root::.bucket.meta.%e1%a0%83%e8%94%89%ed%88%8f%ef%84%8f%e5%98%85%eb%8c%8c%e9%90%8b%e5%98%88%ed%88%8d%ed%88%8d%e5%98%85%e8%94%8b%eb%8c%8d%e8%94%8a%e0%a4%80%3a42bb32be-1e05-491f-815b-a606bdf24d77.4355.613:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e146) 2019-11-29T16:20:28.252 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:20:28.247+0000 1694c700 -1 osd.3 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:47.585859+0000 front 2019-11-29T16:19:47.575824+0000 (oldest deadline 2019-11-29T16:20:28.019488+0000) 2019-11-29T16:20:28.810 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:20:28.793+0000 1694c700 -1 osd.5 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:43.314685+0000 front 2019-11-29T16:19:46.379451+0000 (oldest deadline 2019-11-29T16:20:28.508380+0000) 2019-11-29T16:20:29.104 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:20:29.098+0000 1694c700 -1 osd.6 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:44.636120+0000 front 2019-11-29T16:19:46.403502+0000 (oldest deadline 2019-11-29T16:20:28.709513+0000) 2019-11-29T16:20:29.229 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:20:29.225+0000 1694c700 -1 osd.3 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:47.585859+0000 front 2019-11-29T16:19:47.575824+0000 (oldest deadline 2019-11-29T16:20:28.019488+0000) 2019-11-29T16:20:29.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:20:29.817+0000 1694c700 -1 osd.5 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:43.314685+0000 front 2019-11-29T16:19:46.379451+0000 (oldest deadline 2019-11-29T16:20:28.508380+0000) 2019-11-29T16:20:30.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:20:30.089+0000 1694c700 -1 osd.7 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:47.896724+0000 front 2019-11-29T16:19:46.966968+0000 (oldest deadline 2019-11-29T16:20:30.044531+0000) 2019-11-29T16:20:30.131 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:20:30.124+0000 1694c700 -1 osd.6 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:44.636120+0000 front 2019-11-29T16:19:46.403502+0000 (oldest deadline 2019-11-29T16:20:28.709513+0000) 2019-11-29T16:20:30.217 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:20:30.210+0000 1694c700 -1 osd.3 147 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:19:47.585859+0000 front 2019-11-29T16:19:47.575824+0000 (oldest deadline 2019-11-29T16:20:28.019488+0000) 2019-11-29T16:22:04.979 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:22:04.709+0000 20e22700 -1 RGWWatcher::handle_error cookie 900317632 err (107) Transport endpoint is not connected 2019-11-29T16:22:08.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:08.663+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:09.647 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:09.643+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:10.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:10.652+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:11.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:11.680+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:12.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:12.651+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:13.554 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:13.549+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:13.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:13.664+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:13.673 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:13.669+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:13.839 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:13.834+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:14.089 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:14.083+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:14.126 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:14.121+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:14.578 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:14.564+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:14.647 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:14.641+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:14.647 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:14.643+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:14.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:14.794+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:14.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:14.949+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:15.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:15.099+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:15.108 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:15.101+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:15.588 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:15.583+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:15.602 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:15.597+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:15.604 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:15.600+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:15.760 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:15.754+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:15.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:15.971+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:16.067 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:16.060+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:16.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:16.089+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:16.561 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:16.554+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:16.566 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:16.560+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:16.569 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:16.565+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:16.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:16.757+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:16.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:16.932+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:17.072 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:17.067+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:17.091 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:17.085+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:17.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:17.559+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:17.579 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:17.571+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:17.595 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:17.589+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:17.712 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:17.706+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:17.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:17.765+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:17.950 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:17.944+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:18.055 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:18.050+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:18.117 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:18.110+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:18.587 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:18.580+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:18.595 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:18.588+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:18.595 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:18.591+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:18.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:18.754+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:18.795 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:18.789+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:18.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:18.979+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:19.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:19.068+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:19.165 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:19.158+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:19.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:19.609+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:19.624 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:19.619+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:19.627 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:19.622+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:19.753 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:19.748+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:19.821 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:19.816+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:19.959 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:19.953+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:20.069 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:20.059+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:20.209 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:20.201+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:20.629 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:20.623+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:20.642 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:20.634+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:20.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:20.639+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:20.804 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:20.797+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:20.811 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:20.804+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:20.997 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:20.992+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:21.078 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:21.074+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:21.257 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:21.251+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:21.589 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:21.582+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:21.644 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:21.638+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:21.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:21.642+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:21.802 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:21.794+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:21.857 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:21.851+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:21.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:21.974+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:22.083 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:22.078+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:22.273 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:22.264+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:22.596 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:22:22.589+0000 1694c700 -1 osd.4 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:27.898283+0000 front 2019-11-29T16:21:27.892795+0000 (oldest deadline 2019-11-29T16:22:13.110178+0000) 2019-11-29T16:22:22.680 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:22.674+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:22.681 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:22.677+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:22.828 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:22:22.821+0000 1694c700 -1 osd.6 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:31.949291+0000 front 2019-11-29T16:21:31.946904+0000 (oldest deadline 2019-11-29T16:22:13.216613+0000) 2019-11-29T16:22:22.846 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:22:22.840+0000 1694c700 -1 osd.7 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:32.312392+0000 front 2019-11-29T16:21:31.946915+0000 (oldest deadline 2019-11-29T16:22:17.685270+0000) 2019-11-29T16:22:22.958 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:22:22.954+0000 1694c700 -1 osd.2 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.957239+0000 front 2019-11-29T16:21:31.567072+0000 (oldest deadline 2019-11-29T16:22:14.439878+0000) 2019-11-29T16:22:23.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:22:23.060+0000 1694c700 -1 osd.3 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:28.234082+0000 front 2019-11-29T16:21:28.219793+0000 (oldest deadline 2019-11-29T16:22:13.470394+0000) 2019-11-29T16:22:23.305 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:22:23.296+0000 1694c700 -1 osd.5 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:30.163435+0000 front 2019-11-29T16:21:29.803028+0000 (oldest deadline 2019-11-29T16:22:13.270360+0000) 2019-11-29T16:22:23.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:23.683+0000 1694c700 -1 osd.0 151 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:21:34.582751+0000 front 2019-11-29T16:21:34.838330+0000 (oldest deadline 2019-11-29T16:22:13.473885+0000) 2019-11-29T16:22:23.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:22:23.684+0000 1694c700 -1 osd.0 151 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e150) 2019-11-29T16:22:42.164 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.143+0000 3927b700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.165 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.150+0000 35a74700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.302 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.277+0000 34271700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.409 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.322+0000 34271700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.676 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.495+0000 38a7a700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.693 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.567+0000 35273700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.720 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.602+0000 38279700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:42.794 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.608+0000 33a70700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:43.123 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:42.763+0000 37a78700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:22:43.403 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:22:43.389+0000 38279700 -1 osd.1 156 build_incremental_map_msg missing incremental map 156 2019-11-29T16:23:11.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:23:11.374+0000 1694c700 -1 osd.0 157 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e155) 2019-11-29T16:23:12.519 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:23:12.480+0000 1694c700 -1 osd.0 157 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e155) 2019-11-29T16:23:13.778 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:23:13.742+0000 1694c700 -1 osd.0 157 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e155) 2019-11-29T16:23:23.668 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:36.558 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.672 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:36.562 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.751 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:36.640 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.755 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:36.644 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.779 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:36.554 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.797 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:36.572 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.930 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:36.820 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:23.970 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:36.858 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:24.214 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:36.989 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.230 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:38.119 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.231 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:38.120 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.243 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:10:38.132 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.400 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:38.290 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.401 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:38.291 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.412 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:10:38.302 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.651 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:38.426 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.652 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:38.428 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:23:25.668 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:10:38.443 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:24:05.701 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:05.692+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:05.709 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:05.696+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:05.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:05.742+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:06.118 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:06.114+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:06.661 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:06.652+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:06.673 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:06.669+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:06.719 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:06.712+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:07.132 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:07.127+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:07.653 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:07.648+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:07.670 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:07.666+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:07.678 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:07.672+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:08.147 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:08.143+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:08.535 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:08.530+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:08.613 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:08.607+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:08.668 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:08.663+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:08.676 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:08.670+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:09.162 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:09.153+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:09.181 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:09.175+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:09.521 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:09.511+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:09.636 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:09.629+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:09.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:09.631+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:09.703 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:09.698+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:09.952 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:09.944+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:10.160 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:10.154+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:10.199 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:10.195+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:10.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:10.486+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:10.610 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:10.603+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:10.620 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:10.614+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:10.676 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:10.669+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:10.978 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:10.970+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:11.125 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:11.117+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:11.206 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:11.201+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:11.514 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:11.509+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:11.580 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:11.572+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:11.591 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:11.584+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:11.655 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:11.652+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:11.962 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:11.955+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:12.138 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:12.132+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:12.248 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:12.243+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:12.485 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:12.479+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:12.551 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:12.545+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:12.591 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:12.584+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:12.701 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:12.696+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:12.974 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:12.967+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:13.120 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:13.110+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:13.239 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:13.233+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:13.480 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:13.476+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:13.524 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:13.517+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:13.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:13.640+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:13.736 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:13.729+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:13.999 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:13.992+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:14.166 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:14.158+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:14.228 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:14.223+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:14.456 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:14.450+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:14.483 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:14.477+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:14.638 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:14.632+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:14.735 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:14.731+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:14.993 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:14.986+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:15.219 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:15.211+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:15.248 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:15.245+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:15.467 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:15.460+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:15.470 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:15.466+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:15.652 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:15.646+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:15.695 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:15.690+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:15.980 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:15.973+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:16.215 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:16.209+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:16.224 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:16.220+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:16.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:16.478+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:16.515 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:16.511+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:16.637 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:16.628+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:16.730 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:16.724+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:16.949 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:16.943+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:17.202 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:17.195+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:17.246 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:17.241+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:17.469 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:17.462+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:17.549 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:17.540+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:17.682 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:17.675+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:17.751 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:17.748+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:17.928 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:24:17.921+0000 1694c700 -1 osd.7 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.558772+0000 front 2019-11-29T16:23:24.584648+0000 (oldest deadline 2019-11-29T16:24:09.245169+0000) 2019-11-29T16:24:18.256 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:24:18.251+0000 1694c700 -1 osd.5 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.549294+0000 front 2019-11-29T16:23:23.567799+0000 (oldest deadline 2019-11-29T16:24:09.173288+0000) 2019-11-29T16:24:18.271 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:18.265+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:24:18.445 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:24:18.440+0000 1694c700 -1 osd.6 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.495011+0000 front 2019-11-29T16:23:22.380676+0000 (oldest deadline 2019-11-29T16:24:05.117609+0000) 2019-11-29T16:24:18.593 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:24:18.587+0000 1694c700 -1 osd.3 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:23.624224+0000 front 2019-11-29T16:23:23.613002+0000 (oldest deadline 2019-11-29T16:24:07.674256+0000) 2019-11-29T16:24:18.643 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:24:18.636+0000 1694c700 -1 osd.4 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:24.594539+0000 front 2019-11-29T16:23:24.698198+0000 (oldest deadline 2019-11-29T16:24:05.670226+0000) 2019-11-29T16:24:18.746 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:24:18.738+0000 1694c700 -1 osd.1 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:21.575827+0000 front 2019-11-29T16:23:21.561334+0000 (oldest deadline 2019-11-29T16:24:05.479524+0000) 2019-11-29T16:24:19.240 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:24:19.236+0000 1694c700 -1 osd.2 157 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:23:20.971997+0000 front 2019-11-29T16:23:21.140533+0000 (oldest deadline 2019-11-29T16:24:05.582438+0000) 2019-11-29T16:25:16.118 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:16.111+0000 1694c700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:16.717 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:16.661+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:16.841 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:16.818+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:17.027 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:17.024+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:17.174 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:17.170+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:17.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:17.713+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:17.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:17.913+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:18.049 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:18.044+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:18.179 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:18.173+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:18.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:18.721+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:18.944 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:18.939+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:19.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:19.061+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:19.192 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:19.188+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:19.699 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:19.694+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:19.776 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:25:19.709+0000 20e22700 -1 RGWWatcher::handle_error cookie 363800608 err (107) Transport endpoint is not connected 2019-11-29T16:25:19.958 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:19.952+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:20.073 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:20.068+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:20.198 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:20.194+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:20.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:20.728+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:21.004 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:20.998+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:21.106 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:21.098+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:21.193 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:21.188+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:21.705 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:21.700+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:21.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:21.897+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:22.039 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:22.032+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:22.189 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:22.184+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:22.190 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:22.139+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:22.678 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:22.674+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:22.948 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:22.943+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:23.050 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:23.045+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:23.110 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:23.103+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:23.238 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:23.231+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:23.640 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:23.637+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:23.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:23.980+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:24.028 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:24.019+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:24.152 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:24.139+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:24.288 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:24.279+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:24.643 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:24.637+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:24.950 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:24.944+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:25.035 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:25.027+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:25.195 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:25.189+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:25.266 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:25.259+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:25.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:25.626+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:26.000 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:25.996+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:26.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:26.028+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:26.187 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:26.179+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:26.236 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:26.230+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:26.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:26.667+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:27.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:27.014+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:27.077 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:27.065+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:27.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:27.181+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:27.247 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:27.241+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:27.638 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:27.632+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:28.052 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:28.047+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:28.054 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:28.046+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:28.190 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:28.183+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:28.255 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:25:28.248+0000 1694c700 -1 osd.5 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.984200+0000 front 2019-11-29T16:24:33.477945+0000 (oldest deadline 2019-11-29T16:25:15.755054+0000) 2019-11-29T16:25:28.653 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:28.650+0000 1694c700 -1 osd.3 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:34.745709+0000 front 2019-11-29T16:24:34.813256+0000 (oldest deadline 2019-11-29T16:25:16.169147+0000) 2019-11-29T16:25:29.040 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:25:29.035+0000 1694c700 -1 osd.2 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:35.566397+0000 front 2019-11-29T16:24:35.713773+0000 (oldest deadline 2019-11-29T16:25:20.966796+0000) 2019-11-29T16:25:29.084 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:25:29.078+0000 1694c700 -1 osd.7 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:32.616524+0000 front 2019-11-29T16:24:35.986170+0000 (oldest deadline 2019-11-29T16:25:15.901448+0000) 2019-11-29T16:25:29.206 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:25:29.198+0000 1694c700 -1 osd.4 162 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:24:31.906059+0000 front 2019-11-29T16:24:31.743048+0000 (oldest deadline 2019-11-29T16:25:16.975843+0000) 2019-11-29T16:25:30.913 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:25:30.898+0000 1694c700 -1 osd.3 164 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80542 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-11-29T16:26:15.336 INFO:teuthology.orchestra.run.smithi069.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:26:15.492 INFO:teuthology.orchestra.run.smithi069.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-11-29T16:26:29.903 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:29.893+0000 33a70700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.474 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.359+0000 38a7a700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.475 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.361+0000 38279700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.478 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.382+0000 36a76700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.580 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.523+0000 34271700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.613 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.543+0000 38a7a700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:30.633 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:26:30.585+0000 3a27d700 -1 osd.1 167 build_incremental_map_msg missing incremental map 167 2019-11-29T16:26:59.130 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:26:59.088+0000 1694c700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=10 ondisk+retry+read+known_if_redirected e165) 2019-11-29T16:27:00.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:27:00.578+0000 1694c700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=10 ondisk+retry+read+known_if_redirected e165) 2019-11-29T16:27:32.660 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:14:45.550 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.670 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:14:45.560 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.685 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:14:45.573 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.691 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:14:45.580 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.697 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:14:45.469 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.729 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:14:45.504 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:32.975 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:14:45.865 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:33.047 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:14:45.935 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:33.121 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:14:45.896 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:27:50.980 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:50.974+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:51.183 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:51.179+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:51.200 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:51.193+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:51.992 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:51.987+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:52.102 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:52.096+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:52.191 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:52.187+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:52.201 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:52.195+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:53.024 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:53.017+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:53.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:53.064+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:53.150 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:53.146+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:53.193 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:53.186+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:53.323 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:53.318+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:54.056 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:54.050+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:54.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:54.093+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:54.157 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:54.150+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:54.197 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:54.192+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:54.294 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:54.290+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:54.520 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:54.514+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:27:55.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:55.087+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:55.096 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:55.090+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:55.158 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:55.154+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:55.168 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:55.163+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:55.293 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:55.287+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:55.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:55.540+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:27:56.080 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:56.073+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:56.124 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:56.118+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:56.126 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:27:56.122+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:27:56.153 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:56.148+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:56.154 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:56.147+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:56.280 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:56.275+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:56.521 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:56.514+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:27:57.082 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:57.076+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:57.100 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:27:57.095+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:27:57.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:57.097+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:57.194 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:57.189+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:57.195 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:57.188+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:57.257 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:57.251+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:57.553 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:57.547+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:27:58.046 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:58.040+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:58.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:58.054+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:58.146 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:27:58.141+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:27:58.169 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:58.162+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:58.224 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:58.220+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:58.250 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:58.244+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:58.567 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:58.560+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:27:59.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:27:59.019+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:27:59.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:27:59.097+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:27:59.150 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:27:59.145+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:27:59.209 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:27:59.205+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:27:59.211 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:27:59.205+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:27:59.231 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:27:59.227+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:27:59.566 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:27:59.559+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:00.015 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:00.009+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:00.102 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:00.095+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:00.108 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:00.103+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:00.177 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:00.173+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:00.197 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:00.190+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:00.249 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:00.244+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:00.560 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:00.554+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:01.029 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:01.022+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:01.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:01.081+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:01.144 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:01.139+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:01.196 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:01.190+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:01.203 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:01.199+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:01.285 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:01.280+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:01.519 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:01.513+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:02.056 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:02.049+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:02.091 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:02.085+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:02.108 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:02.103+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:02.166 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:02.159+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:02.176 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:02.172+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:02.279 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:02.273+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:02.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:02.545+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:03.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:03.063+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:03.109 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:03.102+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:03.135 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:03.130+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:03.136 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:03.132+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:03.138 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:03.132+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:03.330 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:03.325+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:03.583 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:03.575+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:04.104 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:04.099+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:04.113 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:04.107+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:04.145 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:04.138+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:04.168 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:04.163+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:04.180 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:04.176+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:04.287 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:04.283+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:04.606 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:04.600+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:05.125 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:05.119+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:05.126 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:05.120+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:05.141 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:05.135+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:05.154 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:05.148+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:05.189 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:05.185+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:05.266 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:05.261+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:05.590 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:05.585+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:06.098 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:06.092+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:06.140 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:06.133+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:06.151 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:06.143+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:06.157 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:06.150+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:06.158 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:06.153+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:06.312 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:06.307+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:06.560 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:06.554+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:07.088 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:07.082+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:07.122 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:07.105+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:07.152 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:07.145+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:07.166 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:07.160+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:07.171 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:07.166+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:07.338 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:07.332+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:07.568 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:07.562+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:08.134 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:08.128+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:08.158 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:08.152+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:08.159 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:08.153+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:08.197 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:08.191+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:08.201 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:08.195+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:08.360 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:08.353+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:08.605 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:08.599+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:09.116 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:09.110+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:09.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:09.111+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:09.189 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:09.182+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:09.198 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:09.192+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:09.199 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:09.193+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:09.401 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:09.395+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:09.604 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:09.597+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:10.092 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:10.086+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:10.103 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:10.098+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:10.203 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:10.198+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:10.215 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:10.209+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:10.248 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:10.241+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:10.384 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:10.377+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:10.561 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:10.555+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:11.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:11.063+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:11.157 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:11.151+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:11.215 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:11.209+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:11.220 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:11.215+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:11.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:11.251+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:11.345 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:11.340+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:11.528 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:11.522+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:12.037 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:12.030+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:12.173 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:12.167+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:12.242 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:12.237+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:12.251 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:12.246+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:12.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:12.251+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:12.333 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:12.326+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:12.533 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:12.522+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:13.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:13.012+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:13.131 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:13.124+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:13.202 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:13.197+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:13.241 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:13.235+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:13.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:13.299+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:13.375 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:13.369+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:13.534 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:13.526+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:28:14.059 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:28:14.052+0000 1694c700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.531171+0000 front 2019-11-29T16:27:10.265127+0000 (oldest deadline 2019-11-29T16:27:55.113678+0000) 2019-11-29T16:28:14.171 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:28:14.165+0000 1694c700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:08.752476+0000 front 2019-11-29T16:27:10.510395+0000 (oldest deadline 2019-11-29T16:27:50.917175+0000) 2019-11-29T16:28:14.219 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:28:14.213+0000 1694c700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.348037+0000 front 2019-11-29T16:27:09.832792+0000 (oldest deadline 2019-11-29T16:27:52.607525+0000) 2019-11-29T16:28:14.243 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:28:14.237+0000 1694c700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.478935+0000 front 2019-11-29T16:27:05.481380+0000 (oldest deadline 2019-11-29T16:27:51.282419+0000) 2019-11-29T16:28:14.326 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:28:14.321+0000 1694c700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.730234+0000 front 2019-11-29T16:27:10.533756+0000 (oldest deadline 2019-11-29T16:27:50.671468+0000) 2019-11-29T16:28:14.408 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:28:14.401+0000 1694c700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:05.276686+0000 front 2019-11-29T16:27:10.637045+0000 (oldest deadline 2019-11-29T16:27:50.555561+0000) 2019-11-29T16:28:14.509 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:28:14.502+0000 1694c700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:27:10.538542+0000 front 2019-11-29T16:27:09.967149+0000 (oldest deadline 2019-11-29T16:27:54.420940+0000) 2019-11-29T16:29:02.265 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:02.260+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:03.340 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:03.276+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:04.263 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:04.258+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:05.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:05.303+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:06.289 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:06.283+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:07.327 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:07.322+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:08.317 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:08.312+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:09.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:09.293+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:10.253 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:10.249+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:11.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:11.256+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:12.042 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:12.035+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:12.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:12.243+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:12.250 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:12.246+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:13.055 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:13.048+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:13.229 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:13.223+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:13.230 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:13.226+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:13.244 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:29:13.037+0000 20e22700 -1 RGWWatcher::handle_error cookie 5473376496 err (107) Transport endpoint is not connected 2019-11-29T16:29:13.569 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:13.562+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:13.874 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:13.867+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:14.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:14.088+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:14.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:14.226+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:14.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:14.228+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:14.562 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:14.555+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:14.855 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:14.848+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:15.121 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:15.112+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:15.255 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:15.248+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:15.256 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:15.252+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:15.564 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:15.558+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:15.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:15.680+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:15.810 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:15.803+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:15.851 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:15.844+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:16.120 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:16.113+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:16.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:16.282+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:16.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:16.284+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:16.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:16.609+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:16.712 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:16.692+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:16.777 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:16.770+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:16.893 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:16.886+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:17.134 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:17.126+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:17.315 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:17.311+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:17.316 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:17.312+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:17.666 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:17.657+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:17.673 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:17.668+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:17.793 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:17.787+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:17.866 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:17.860+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:18.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:18.092+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:18.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:18.309+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:18.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:18.311+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:18.653 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:18.643+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:18.673 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:18.665+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:18.836 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:18.828+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:18.851 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:18.844+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:19.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:19.133+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:19.345 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:19.340+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:19.347 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:19.343+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:19.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:19.596+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:19.657 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:19.650+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:19.889 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:19.883+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:19.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:19.891+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:20.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:20.111+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:20.399 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:20.394+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:20.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:20.396+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:20.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:20.584+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:20.619 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:20.613+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:20.862 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:20.856+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:20.885 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:20.880+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:21.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:21.131+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:21.380 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:21.376+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:21.381 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:21.377+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:21.586 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:21.580+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:21.600 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:21.595+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:21.845 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:21.839+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:21.907 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:21.902+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:22.110 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:22.104+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:22.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:22.345+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:22.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:22.347+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:22.566 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:22.560+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:22.579 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:22.575+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:22.832 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:22.819+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:22.939 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:22.933+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:23.150 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:23.143+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:23.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:23.397+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:23.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:23.399+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:23.590 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:23.585+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:23.607 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:23.600+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:23.818 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:23.812+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:23.971 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:23.966+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:24.125 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:24.120+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:24.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:24.397+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:24.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:24.400+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:24.573 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:24.568+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:24.633 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:24.622+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:24.787 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:24.782+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:24.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:24.956+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:25.163 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:25.158+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:25.452 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:25.446+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:25.453 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:25.449+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:25.576 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:25.570+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:25.653 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:25.647+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:25.809 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:25.804+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:25.952 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:25.943+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:26.138 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:26.130+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:26.431 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:26.427+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:26.433 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:26.430+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:26.628 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:26.622+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:26.660 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:26.655+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:26.811 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:26.806+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:26.959 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:26.953+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:27.167 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:27.162+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:27.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:27.422+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:27.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:27.424+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:27.621 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:27.614+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:27.665 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:27.659+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:27.835 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:27.830+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:27.939 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:27.932+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:28.132 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:28.126+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:28.472 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:28.466+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:28.473 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:28.467+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:28.625 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:28.619+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:28.700 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:28.696+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:28.885 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:28.872+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:28.957 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:28.951+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:29.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:29.131+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:29.427 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:29.422+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:29.430 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:29.427+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:29.617 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:29.611+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:29.688 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:29.683+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:29.922 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:29.917+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:29.938 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:29.932+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:30.190 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:30.184+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:30.202 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:30.199+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:30.465 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:30.461+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:30.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:30.464+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:30.643 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:30.637+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:30.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:30.691+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:30.937 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:30.927+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:30.949 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:30.943+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:31.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:31.184+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:31.223 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:31.219+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:31.513 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:31.507+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:31.514 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:31.509+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:31.655 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:31.649+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:31.707 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:31.692+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:31.940 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:31.935+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:31.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:31.946+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:32.220 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:32.213+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:32.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:32.259+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:32.493 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:32.488+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:32.494 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:32.490+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:32.664 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:32.658+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:32.734 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:32.728+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:32.970 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:32.964+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:33.000 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:32.994+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:33.192 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:33.186+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:33.223 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:33.218+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:33.531 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:33.526+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:33.534 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:33.530+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:33.681 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:33.675+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:33.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:33.748+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:33.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:33.969+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:34.007 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:33.999+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:34.183 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:29:34.171+0000 1694c700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:26.040861+0000 front 2019-11-29T16:28:26.046351+0000 (oldest deadline 2019-11-29T16:29:11.931176+0000) 2019-11-29T16:29:34.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:34.190+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:34.521 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:34.517+0000 1694c700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.806844+0000 front 2019-11-29T16:28:26.725940+0000 (oldest deadline 2019-11-29T16:29:11.786890+0000) 2019-11-29T16:29:34.523 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:34.520+0000 1694c700 -1 osd.3 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:34.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:29:34.654+0000 1694c700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:28.349262+0000 front 2019-11-29T16:28:27.737624+0000 (oldest deadline 2019-11-29T16:29:12.647333+0000) 2019-11-29T16:29:34.735 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:29:34.731+0000 1694c700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:29.539127+0000 front 2019-11-29T16:28:29.592843+0000 (oldest deadline 2019-11-29T16:29:15.101437+0000) 2019-11-29T16:29:34.945 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:29:34.939+0000 1694c700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:31.369564+0000 front 2019-11-29T16:28:31.792255+0000 (oldest deadline 2019-11-29T16:29:13.471231+0000) 2019-11-29T16:29:34.998 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:29:34.993+0000 1694c700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:28:30.457147+0000 front 2019-11-29T16:28:30.537337+0000 (oldest deadline 2019-11-29T16:29:15.568380+0000) 2019-11-29T16:29:35.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:29:35.202+0000 1694c700 -1 osd.0 172 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e171) 2019-11-29T16:29:35.664 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:35.646+0000 1694c700 -1 osd.3 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:29:36.729 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:29:36.666+0000 1694c700 -1 osd.3 174 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:97592 2.28s0 2:17f5fd80:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%e0%a4%80%e9%90%8b%e2%98%83%ef%84%91%e9%90%8b%e5%98%88%e8%94%8b%e5%98%88%ef%84%91%e8%94%8a%eb%8c%8d%ce%a9%e1%a0%84%e1%a0%81%ed%88%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-11-29T16:30:24.337 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.318+0000 36275700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.346 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.322+0000 3a27d700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.529 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.483+0000 34a72700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.547 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.516+0000 38279700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.606 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.568+0000 38279700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.652 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.488+0000 34271700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:24.663 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:24.603+0000 38279700 -1 osd.1 176 build_incremental_map_msg missing incremental map 176 2019-11-29T16:30:30.873 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:30:30.868+0000 39a7c700 -1 osd.1 177 build_incremental_map_msg missing incremental map 176 2019-11-29T16:31:32.798 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:45.687 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:32.805 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:45.695 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:32.819 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:45.708 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:32.821 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:45.710 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:32.839 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:45.614 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:32.848 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:45.623 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:33.037 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:45.926 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:33.066 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:45.955 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:33.161 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:45.935 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.496 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:58.385 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.514 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:58.404 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.544 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:58.432 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.546 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:18:58.435 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.577 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:58.351 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.582 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:58.471 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.588 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:58.363 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.624 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:18:58.511 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:45.641 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:18:58.414 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:31:46.206 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:46.199+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:47.197 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:47.190+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:48.185 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:48.178+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:48.329 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:48.319+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:48.329 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:48.294+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:48.846 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:48.840+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:49.238 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:49.232+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:49.289 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:49.282+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:49.330 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:49.322+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:49.349 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:49.342+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:49.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:49.763+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:49.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:49.864+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:50.265 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:50.258+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:50.285 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:50.276+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:50.371 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:50.366+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:50.379 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:50.373+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:50.810 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:50.803+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:50.869 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:50.863+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:50.939 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:50.934+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:51.265 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:51.258+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:51.307 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:51.300+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:51.351 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:51.345+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:51.384 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:51.377+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:51.857 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:51.852+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:51.862 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:51.856+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:51.939 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:51.933+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:52.297 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:52.291+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:52.326 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:52.321+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:52.348 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:52.343+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:52.366 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:52.359+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:52.858 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:52.852+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:52.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:52.891+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:52.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:52.945+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:53.270 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:53.265+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:53.333 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:53.328+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:53.341 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:53.334+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:53.388 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:53.383+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:53.845 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:53.840+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:53.918 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:53.911+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:53.985 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:53.980+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:54.312 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:54.307+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:54.344 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:54.338+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:54.348 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:54.341+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:54.349 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:54.343+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:54.892 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:54.887+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:54.911 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:54.905+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:54.958 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:54.953+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:55.301 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:55.296+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:55.331 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:55.325+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:55.363 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:55.357+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:55.375 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:55.369+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:55.875 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:55.865+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:55.932 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:55.925+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:55.970 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:55.962+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:56.286 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:56.279+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:56.300 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:56.295+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:56.329 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:56.324+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:56.404 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:56.398+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:56.885 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:56.878+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:56.906 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:56.900+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:56.939 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:56.933+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:57.288 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:57.283+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:57.327 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:57.322+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:57.358 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:57.353+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:57.401 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:57.395+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:57.843 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:57.837+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:57.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:57.904+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:57.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:57.908+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:31:57.941 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:57.930+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:58.302 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:58.292+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:58.330 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:58.323+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:58.345 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:58.339+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:58.395 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:58.389+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:58.879 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:58.873+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:58.909 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:58.902+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:58.910 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:58.905+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:31:58.922 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:58.916+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:31:59.264 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:31:59.257+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:31:59.376 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:31:59.370+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:31:59.398 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:31:59.393+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:31:59.413 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:31:59.407+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:31:59.874 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:59.867+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:31:59.881 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:31:59.872+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:31:59.919 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:31:59.912+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:31:59.973 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:31:59.968+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:00.243 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:00.236+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:00.409 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:00.404+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:00.420 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:00.414+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:00.446 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:00.440+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:00.900 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:00.893+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:00.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:00.908+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:00.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:00.909+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:00.952 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:00.946+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:01.204 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:01.199+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:01.384 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:01.377+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:01.467 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:01.461+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:01.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:01.494+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:01.891 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:01.884+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:01.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:01.918+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:01.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:01.920+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:01.988 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:01.983+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:02.252 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:02.247+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:02.410 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:02.404+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:02.501 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:02.494+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:02.508 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:02.502+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:02.855 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:02.849+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:02.905 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:02.899+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:02.906 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:02.902+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:03.044 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:03.030+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:03.267 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:03.261+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:03.437 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:03.431+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:03.499 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:03.493+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:03.503 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:03.498+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:03.853 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:03.847+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:03.902 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:03.897+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:03.903 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:03.898+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:04.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:04.032+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:04.274 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:04.267+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:04.463 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:04.457+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:04.519 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:04.513+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:04.536 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:04.530+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:04.821 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:04.815+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:04.945 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:04.939+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:04.946 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:04.941+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:05.041 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:05.034+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:05.329 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:05.321+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:05.461 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:05.455+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:05.524 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:05.519+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:05.532 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:05.526+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:05.781 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:05.775+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:05.980 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:05.970+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:05.980 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:05.973+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:06.022 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:06.015+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:06.319 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:06.313+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:06.477 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:06.465+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:06.499 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:06.492+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:06.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:06.549+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:06.800 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:06.795+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:06.939 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:06.932+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:06.942 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:06.937+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:06.987 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:06.979+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:07.351 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:07.344+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:07.440 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:07.434+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:07.485 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:07.478+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:07.575 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:07.570+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:07.825 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:07.816+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:07.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:07.968+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:07.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:07.971+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:08.019 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:08.012+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:08.355 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:08.349+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:08.437 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:08.431+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:08.478 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:08.473+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:08.587 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:08.580+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:08.808 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:08.802+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:09.028 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:09.022+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:09.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:09.024+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:09.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:09.051+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:09.338 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:09.332+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:09.427 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:09.421+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:09.493 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:09.486+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:09.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:09.620+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:09.777 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:09.771+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:10.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:10.005+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:10.014 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:10.008+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:10.053 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:10.047+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:10.336 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:10.329+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:10.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:10.408+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:10.519 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:10.513+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:10.661 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:10.656+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:10.741 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:10.735+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:11.023 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:11.018+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:11.027 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:11.021+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:11.034 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:11.028+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:11.376 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:11.369+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:11.401 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:11.396+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:11.488 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:11.477+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:11.635 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:11.625+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:11.739 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:11.734+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:12.053 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:12.047+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:12.054 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:12.048+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:12.054 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:12.049+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:12.410 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:12.404+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:12.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:12.417+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:12.474 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:12.468+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:12.611 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:12.605+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:12.711 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:12.706+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:13.041 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:13.035+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:13.092 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:13.086+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:13.093 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:13.088+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:13.393 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:13.387+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:13.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:13.416+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:13.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:13.480+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:13.598 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:13.592+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:13.690 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:13.685+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:14.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:14.042+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:14.049 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:14.046+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:14.073 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:14.067+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:14.395 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:14.390+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:14.399 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:14.393+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:14.506 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:14.501+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:14.636 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:14.629+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:14.744 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:14.737+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:15.054 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:15.049+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:15.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:15.053+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:15.113 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:15.107+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:15.376 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:15.370+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:15.401 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:15.391+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:15.557 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:15.552+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:15.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:15.665+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:15.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:15.694+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:16.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:16.049+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:16.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:16.051+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:16.110 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:16.099+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:16.352 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:16.346+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:16.436 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:16.430+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:16.523 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:16.517+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:16.697 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:16.692+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:16.706 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:16.698+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:17.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:17.089+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:17.096 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:17.091+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:17.105 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:17.099+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:17.349 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:17.344+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:17.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:17.418+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:17.485 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:17.479+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:17.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:17.666+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:17.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:17.690+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:18.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:18.053+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:18.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:18.055+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:18.092 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:18.087+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:18.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:18.383+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:18.391 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:18.387+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:18.503 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:18.497+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:18.665 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:18.660+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:18.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:18.717+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:19.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:19.012+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:19.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:19.015+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:19.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:19.090+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:19.373 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:19.366+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:19.404 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:19.399+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:19.470 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:19.464+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:19.658 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:19.653+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:19.737 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:19.731+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:20.061 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:20.054+0000 1694c700 -1 osd.3 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.536713+0000 front 2019-11-29T16:31:05.610444+0000 (oldest deadline 2019-11-29T16:31:50.013021+0000) 2019-11-29T16:32:20.063 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:20.060+0000 1694c700 -1 osd.3 177 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:20.083 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:32:20.076+0000 1694c700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:03.507368+0000 front 2019-11-29T16:31:03.309882+0000 (oldest deadline 2019-11-29T16:31:47.855021+0000) 2019-11-29T16:32:20.358 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:32:20.354+0000 1694c700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.584768+0000 front 2019-11-29T16:31:04.748552+0000 (oldest deadline 2019-11-29T16:31:47.704876+0000) 2019-11-29T16:32:20.412 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:32:20.404+0000 1694c700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.586302+0000 front 2019-11-29T16:31:04.683567+0000 (oldest deadline 2019-11-29T16:31:47.765451+0000) 2019-11-29T16:32:20.440 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:20.433+0000 1694c700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.115302+0000 front 2019-11-29T16:31:04.101318+0000 (oldest deadline 2019-11-29T16:31:48.716249+0000) 2019-11-29T16:32:20.598 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:32:20.584+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 46sec e177 v177) 2019-11-29T16:32:20.616 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:32:20.612+0000 1694c700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:04.369479+0000 front 2019-11-29T16:31:04.376499+0000 (oldest deadline 2019-11-29T16:31:49.504112+0000) 2019-11-29T16:32:20.695 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:32:20.689+0000 1694c700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:31:05.602050+0000 front 2019-11-29T16:31:00.305171+0000 (oldest deadline 2019-11-29T16:31:45.569860+0000) 2019-11-29T16:32:21.210 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:21.192+0000 1694c700 -1 osd.3 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:22.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:22.373+0000 1694c700 -1 osd.3 179 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98049 2.19s0 2:9a53a9e9:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ed%88%8f%e8%94%8b%eb%8c%8b%e5%98%88%e0%a4%80%eb%8c%8d%e3%9c%86%e5%98%88%e9%90%8b%e1%a0%82%e9%90%8a%e5%98%88%e3%9c%84%eb%8c%8e%ed%88%8f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e177) 2019-11-29T16:32:30.894 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:19:43.783 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:30.921 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:19:43.810 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:30.949 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:19:43.838 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:30.979 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:19:43.868 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:30.999 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:19:43.771 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:30.999 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:19:43.774 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:31.021 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:19:43.905 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:31.059 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:19:43.947 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:31.136 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:19:43.911 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:32:57.293 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:57.270+0000 34271700 -1 osd.6 181 build_incremental_map_msg missing incremental map 181 2019-11-29T16:32:57.393 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:57.380+0000 38279700 -1 osd.3 181 build_incremental_map_msg missing incremental map 181 2019-11-29T16:32:57.617 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:57.450+0000 38279700 -1 osd.3 181 build_incremental_map_msg missing incremental map 181 2019-11-29T16:32:57.622 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:32:57.596+0000 3b27f700 -1 osd.6 181 build_incremental_map_msg missing incremental map 181 2019-11-29T16:32:57.649 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:57.451+0000 39a7c700 -1 osd.3 181 build_incremental_map_msg missing incremental map 181 2019-11-29T16:32:59.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:32:59.172+0000 11cac700 -1 osd.3 182 build_incremental_map_msg missing incremental map 181 2019-11-29T16:33:00.068 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:00.062+0000 11cac700 -1 osd.6 182 build_incremental_map_msg missing incremental map 181 2019-11-29T16:33:21.915 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:33:21.911+0000 20e22700 -1 RGWWatcher::handle_error cookie 5413978496 err (107) Transport endpoint is not connected 2019-11-29T16:33:22.234 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:22.226+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:23.288 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:23.285+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:24.302 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:24.298+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:25.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:25.305+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:26.270 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:26.266+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:27.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:27.289+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:27.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:27.744+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:28.338 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:28.333+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:28.776 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:28.771+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:29.361 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:29.354+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:29.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:29.763+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:30.363 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:30.359+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:30.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:30.779+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:31.393 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:31.388+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:31.812 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:31.805+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:32.434 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:32.429+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:32.776 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:32.771+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:33.407 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:33.402+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:33.427 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:33:33.214+0000 20e22700 -1 RGWWatcher::handle_error cookie 820951488 err (107) Transport endpoint is not connected 2019-11-29T16:33:33.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:33.767+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:34.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:34.374+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:34.736 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:34.731+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:35.413 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:35.409+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:35.700 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:35.694+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:36.430 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:36.424+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:36.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:36.688+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:37.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:37.387+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:37.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:37.680+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:38.106 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:38.100+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:38.355 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:38.351+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:38.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:38.638+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:39.116 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:39.111+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:39.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:39.354+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:39.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:39.664+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:40.129 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:40.124+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:40.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:40.376+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:40.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:40.668+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:41.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:41.146+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:41.389 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:41.384+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:41.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:41.668+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:42.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:42.180+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:42.396 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:42.390+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:42.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:42.682+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:43.178 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:43.172+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:43.377 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:43.367+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:43.700 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:43.695+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:44.113 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:44.104+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:44.132 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:44.128+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:44.365 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:44.359+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:44.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:44.656+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:45.083 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:45.077+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:45.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:45.108+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:45.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:45.379+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:45.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:45.651+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:46.093 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:46.089+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:46.126 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:46.123+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:46.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:46.388+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:46.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:46.648+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:47.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:47.099+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:47.145 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:47.139+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:47.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:47.343+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:47.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:47.693+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:48.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:48.101+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:48.122 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:48.118+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:48.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:48.314+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:48.680 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:48.675+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:49.147 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:49.142+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:49.162 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:49.157+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:49.365 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:49.361+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:49.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:49.687+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:50.146 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:50.141+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:50.184 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:50.177+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:50.364 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:50.361+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:50.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:50.651+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:51.119 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:51.115+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:51.136 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:51.133+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:51.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:51.338+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:51.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:51.638+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:52.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:52.092+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:52.128 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:52.124+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:52.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:52.372+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:52.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:52.668+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:53.057 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:53.053+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:53.153 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:53.149+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:53.339 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:53.335+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:53.630 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:53.626+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:54.043 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:54.039+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:54.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:54.186+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:54.301 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:54.297+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:54.597 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:54.590+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:55.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:55.082+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:55.208 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:55.204+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:55.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:55.338+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:55.596 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:55.591+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:56.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:56.038+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:56.200 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:56.196+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:56.346 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:56.342+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:56.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:56.639+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:57.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:57.043+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:57.183 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:57.179+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:57.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:57.338+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:57.675 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:57.670+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:58.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:58.021+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:58.143 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:58.139+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:58.395 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:58.391+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:58.636 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:58.632+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:33:59.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:33:59.063+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:59.164 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:33:59.160+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:33:59.401 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:33:59.398+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:33:59.682 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:33:59.678+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:00.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:00.024+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:00.143 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:00.137+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:00.365 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:00.361+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:00.661 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:00.657+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:01.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:01.031+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:01.173 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:01.168+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:01.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:01.387+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:01.642 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:01.639+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:02.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:02.024+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:02.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:02.180+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:02.358 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:02.353+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:02.671 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:02.667+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:03.076 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:03.071+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:03.140 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:03.135+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:03.406 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:03.400+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:03.488 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:03.482+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:03.642 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:03.638+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:04.094 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:04.090+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:04.174 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:04.170+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:04.382 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:04.379+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:04.487 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:04.482+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:04.630 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:04.626+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:05.124 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:05.119+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:05.227 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:05.220+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:05.227 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:05.223+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:05.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:05.410+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:05.418 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:05.412+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:05.519 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:05.513+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:05.601 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:05.594+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:05.602 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:05.599+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:05.782 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:05.776+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:06.160 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:06.156+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:06.279 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:06.273+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:06.282 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:06.278+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:06.412 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:06.407+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:06.413 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:06.409+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:06.552 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:06.539+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:06.633 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:06.629+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:06.635 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:06.631+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:06.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:06.748+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:07.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:07.138+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:07.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:07.140+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:07.317 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:07.307+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:07.317 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:07.310+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:07.375 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:07.370+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:07.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:07.371+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:07.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:07.549+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:07.589 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:07.584+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:07.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:07.588+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:07.737 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:07.729+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:08.191 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:08.186+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:08.192 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:08.188+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:08.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:08.276+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:08.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:08.278+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:08.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:08.377+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:08.382 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:08.378+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:08.549 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:08.544+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:08.563 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:08.558+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:08.567 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:08.563+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:08.725 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:08.718+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:08.890 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:08.885+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:09.152 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:09.146+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:09.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:09.148+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:09.322 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:09.315+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:09.322 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:09.318+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:09.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:09.377+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:09.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:09.380+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:09.524 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:09.517+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:09.524 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:09.520+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:09.569 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:09.563+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:09.746 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:09.740+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:09.885 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:09.879+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:10.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:10.183+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:10.190 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:10.187+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:10.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:10.278+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:10.285 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:10.281+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:10.375 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:10.367+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:10.375 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:10.370+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:10.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:10.487+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:10.496 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:10.491+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:10.564 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:10.557+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:10.723 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:10.716+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:10.903 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:10.895+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:11.217 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:11.207+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:11.217 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:11.211+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:11.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:11.325+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:11.334 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:11.330+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:11.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:11.382+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:11.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:11.384+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:11.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:11.450+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:11.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:11.452+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:11.522 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:11.516+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:11.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:11.760+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:11.932 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:11.927+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:12.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:12.183+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:12.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:12.185+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:12.304 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:12.299+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:12.306 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:12.303+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:12.412 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:12.406+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:12.413 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:12.408+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:12.503 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:12.496+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:12.505 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:12.499+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:12.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:12.550+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:12.821 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:12.811+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:12.887 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:12.882+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:13.175 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:13.169+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:13.178 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:13.173+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:13.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:13.353+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:13.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:13.361+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:13.466 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:13.459+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:13.467 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:13.462+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:13.476 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:13.470+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:13.481 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:13.476+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:13.526 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:13.520+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:13.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:13.841+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:13.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:13.928+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:14.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:14.187+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:14.197 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:14.192+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:14.379 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:14.373+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:14.381 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:14.377+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:14.434 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:14.427+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:14.436 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:14.430+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:14.447 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:14.441+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:14.449 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:14.444+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:14.539 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:14.533+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:14.852 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:14.845+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:14.956 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:14.949+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:15.226 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:15.218+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:15.227 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:15.221+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:15.437 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:15.430+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:15.437 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:15.432+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:15.469 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:15.463+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:15.479 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:15.465+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:15.483 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:15.478+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:15.485 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:15.481+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:15.515 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:15.508+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:15.846 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:15.839+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:15.997 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:15.991+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:16.281 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:16.275+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:16.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:16.281+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:16.435 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:16.429+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:16.440 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:16.434+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:16.457 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:16.450+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:16.458 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:16.452+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:16.479 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:16.473+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:16.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:16.477+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:16.529 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:16.522+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:16.849 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:16.842+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:17.030 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:17.023+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:17.259 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:17.248+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:17.260 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:17.253+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:17.400 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:17.392+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:17.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:17.396+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:17.414 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:17.407+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:17.415 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:17.410+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:17.442 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:17.436+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:17.443 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:17.438+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:17.486 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:17.479+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:17.900 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:17.893+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:18.011 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:18.004+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:18.235 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:18.228+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:18.236 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:18.230+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:18.384 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:18.374+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:18.388 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:18.378+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:18.404 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:18.398+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:18.407 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:18.402+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:18.411 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:18.406+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:18.411 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:18.407+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:18.540 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:18.529+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:18.888 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:18.881+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:19.048 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:19.041+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:19.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:19.281+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:19.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:19.283+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:19.422 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:19.416+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:19.425 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:19.420+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:19.433 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:19.427+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:19.441 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:19.430+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:19.463 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:19.457+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:19.465 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:19.461+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:19.517 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:19.510+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:19.925 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:19.914+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:20.103 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:20.096+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:20.284 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:20.276+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:20.285 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:20.279+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:20.400 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:20.393+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:20.400 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:20.395+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:20.472 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:20.465+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:20.474 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:20.468+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:20.504 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:20.498+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:20.504 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:20.499+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:20.561 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:20.555+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:20.883 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:20.877+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:21.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:21.130+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:21.268 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:21.263+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:21.270 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:21.266+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:21.365 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:21.358+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:21.367 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:21.361+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:21.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:21.451+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:21.459 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:21.455+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:21.462 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:21.456+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:21.465 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:21.460+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:21.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:21.532+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:21.917 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:21.911+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:22.175 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:22.168+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:22.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:22.298+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:22.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:22.299+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:22.339 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:22.332+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:22.340 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:22.335+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:22.412 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:22.407+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:22.415 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:22.411+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:22.502 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:22.494+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:22.502 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:22.496+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:22.509 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:22.503+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:22.878 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:22.870+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:23.225 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:23.220+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:23.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:23.282+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:23.291 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:23.286+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:23.342 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:23.336+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:23.343 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:23.338+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:23.383 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:23.377+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:23.386 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:23.381+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:23.456 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:23.449+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:23.457 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:23.452+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:23.482 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:23.476+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:23.861 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:23.855+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:24.192 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:24.185+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.510152+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:24.250 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:24.244+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:24.252 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:24.247+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:24.382 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:24.374+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:24.383 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:24.377+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:24.426 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:24.419+0000 1694c700 -1 osd.0 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:24.427 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:24.422+0000 1694c700 -1 osd.0 182 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:24.440 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:24.434+0000 1694c700 -1 osd.2 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:33:21.678473+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:24.440 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:24.436+0000 1694c700 -1 osd.2 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:24.473 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:24.468+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:24.878 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:24.872+0000 1694c700 -1 osd.4 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:25.146 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:25.141+0000 1694c700 -1 osd.7 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:34:24.677505+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:25.257 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:25.252+0000 1694c700 -1 osd.3 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.017466+0000 front 2019-11-29T16:33:21.822698+0000 (oldest deadline 2019-11-29T16:34:06.419887+0000) 2019-11-29T16:34:25.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:34:25.254+0000 1694c700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:98227 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820951488 gen 55] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:25.384 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:25.377+0000 1694c700 -1 osd.6 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.675190+0000 front 2019-11-29T16:33:21.596852+0000 (oldest deadline 2019-11-29T16:34:04.327639+0000) 2019-11-29T16:34:25.386 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:25.382+0000 1694c700 -1 osd.6 182 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:25.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:25.411+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:21.783948+0000 front 2019-11-29T16:34:25.314124+0000 (oldest deadline 2019-11-29T16:34:05.175734+0000) 2019-11-29T16:34:25.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:34:25.412+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98197 2.10s0 2:0b4c7bfe:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.613_%ef%84%90%e1%a0%84%eb%8c%8c%e0%a4%80%ed%88%8e%e3%9c%85%eb%8c%8c%e3%9c%86%ed%88%8f%e9%90%89%e2%98%83%e5%98%87%e9%90%8a%ed%88%8f%e0%a4%81:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e180) 2019-11-29T16:34:25.445 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:25.434+0000 1694c700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:25.445 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:25.436+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:25.470 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:34:25.463+0000 1694c700 -1 osd.5 182 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:20.432159+0000 front 2019-11-29T16:33:21.002616+0000 (oldest deadline 2019-11-29T16:34:03.293529+0000) 2019-11-29T16:34:25.912 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:34:25.907+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:19.502374+0000 front 2019-11-29T16:33:19.500195+0000 (oldest deadline 2019-11-29T16:34:05.294825+0000) 2019-11-29T16:34:26.150 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:34:26.106+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:34:24.677505+0000 front 2019-11-29T16:33:22.515165+0000 (oldest deadline 2019-11-29T16:34:08.349892+0000) 2019-11-29T16:34:26.401 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:34:26.398+0000 1694c700 -1 osd.6 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98240 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e182) 2019-11-29T16:34:26.499 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:26.449+0000 1694c700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:26.663 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:26.654+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:27.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:27.705+0000 1694c700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:33:22.220311+0000 front 2019-11-29T16:33:22.495107+0000 (oldest deadline 2019-11-29T16:34:05.410138+0000) 2019-11-29T16:34:27.715 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:27.708+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:28.702 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:28.697+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:29.756 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:29.752+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:30.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:30.795+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:31.781 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:31.776+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:32.764 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:32.759+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:33.819 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:33.814+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:34.804 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:34.799+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:35.771 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:35.767+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:36.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:36.767+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:37.803 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:37.798+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:38.783 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:38.778+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:39.849 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:39.842+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:40.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:40.853+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:41.906 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:41.902+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:42.906 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:42.901+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:43.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:43.949+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:44.951 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:44.947+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:45.940 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:45.935+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:46.989 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:46.984+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:48.005 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:48.000+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:48.986 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:48.982+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:50.030 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:50.025+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:51.025 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:51.020+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:52.035 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:52.031+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:53.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:53.082+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:54.085 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:54.080+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:55.057 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:55.053+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:56.053 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:56.048+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:57.527 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:57.237+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:34:58.335 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:34:58.302+0000 1694c700 -1 osd.0 183 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=16 ondisk+retry+read+known_if_redirected e181) 2019-11-29T16:35:27.585 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:22:40.474 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.593 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:22:40.482 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.620 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:22:40.494 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.620 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:22:40.497 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.620 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:22:40.378 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.620 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:22:40.383 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.836 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:22:40.724 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.838 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:22:40.612 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:27.840 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:22:40.730 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:35:52.979 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:52.973+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:53.485 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:53.480+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:53.662 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:53.655+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:54.015 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:54.010+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:54.409 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:54.403+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:54.531 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:54.525+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:54.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:54.694+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:55.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:55.055+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:55.457 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:55.449+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:55.482 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:35:55.477+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:35:55.505 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:55.498+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:55.677 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:55.670+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:55.786 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:55.778+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:35:55.960 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:35:55.955+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:35:56.095 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:56.082+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:56.413 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:56.408+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:56.489 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:56.483+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:56.518 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:35:56.513+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:35:56.681 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:56.674+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:56.803 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:56.798+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:35:56.951 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:35:56.944+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:35:57.109 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:57.103+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:57.432 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:57.426+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:57.476 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:57.469+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:57.555 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:35:57.549+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:35:57.657 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:57.650+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:57.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:57.757+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:35:57.945 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:35:57.939+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:35:58.099 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:58.093+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:58.446 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:58.440+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:58.523 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:58.517+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:58.546 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:35:58.540+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:35:58.642 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:58.635+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:58.762 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:58.754+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:35:58.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:58.758+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:35:58.981 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:35:58.975+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:35:59.104 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:35:59.098+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:35:59.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:35:59.465+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:35:59.513 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:35:59.507+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:35:59.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:35:59.507+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:35:59.693 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:35:59.686+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:35:59.737 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:59.730+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:35:59.738 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:35:59.733+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:00.007 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:00.000+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:00.122 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:00.116+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:00.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:00.421+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:00.530 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:00.523+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:00.530 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:00.524+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:00.676 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:00.671+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:00.777 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:00.772+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:00.778 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:00.774+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:00.992 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:00.987+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:01.121 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:01.115+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:01.479 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:01.473+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:01.497 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:01.492+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:01.570 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:01.565+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:01.644 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:01.639+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:01.794 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:01.789+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:01.795 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:01.792+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:01.992 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:01.987+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:02.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:02.097+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:02.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:02.465+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:02.517 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:02.505+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:02.551 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:02.545+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:02.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:02.623+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:02.812 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:02.803+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:02.812 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:02.805+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:03.042 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:03.037+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:03.145 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:03.139+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:03.435 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:03.429+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:03.503 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:03.497+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:03.586 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:03.580+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:03.672 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:03.667+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:03.818 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:03.813+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:03.818 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:03.815+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:04.033 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:04.027+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:04.192 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:04.188+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:04.424 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:04.417+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:04.558 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:04.552+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:04.566 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:04.561+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:04.690 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:04.685+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:04.846 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:04.840+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:04.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:04.843+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:05.051 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:05.045+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:05.163 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:05.146+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:05.448 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:05.442+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:05.589 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:05.583+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:05.609 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:05.603+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:05.725 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:05.717+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:05.838 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:05.833+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:05.840 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:05.836+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:06.092 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:06.086+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:06.189 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:06.182+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:06.469 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:06.463+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:06.558 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:06.553+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:06.650 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:06.644+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:06.769 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:06.765+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:06.833 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:06.828+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:06.833 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:06.829+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:07.129 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:07.124+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:07.149 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:07.142+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:07.465 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:07.459+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:07.568 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:07.561+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:07.694 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:07.687+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:07.779 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:07.774+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:07.800 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:07.795+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:07.802 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:07.799+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:08.102 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:08.097+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:08.173 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:08.166+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:08.510 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:08.503+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:08.555 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:08.548+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:08.684 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:08.677+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:08.797 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:08.786+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:08.810 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:08.800+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:08.810 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:08.801+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:09.120 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:09.115+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:09.171 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:09.164+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:09.556 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:09.550+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:09.593 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:09.586+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:09.675 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:09.668+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:09.806 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:09.801+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:09.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:09.802+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:09.807 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:09.803+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:10.137 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:10.130+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:10.157 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:10.151+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:10.551 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:10.544+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:10.589 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:10.581+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:10.708 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:10.702+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:10.770 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:10.766+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:10.770 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:10.767+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:10.846 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:10.841+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:11.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:11.101+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:11.210 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:11.205+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:11.591 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:11.584+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:11.627 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:11.621+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:11.665 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:11.658+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:11.766 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:11.761+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:11.768 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:11.765+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:11.852 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:11.848+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:12.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:12.089+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:12.199 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:12.194+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:12.589 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:12.579+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:12.645 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:12.638+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:12.680 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:12.673+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:12.759 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:12.753+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:12.760 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:12.756+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:12.819 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:12.814+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:13.100 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:13.092+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:13.164 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:13.158+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:13.607 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:13.593+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:13.694 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:13.687+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:13.715 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:13.708+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:13.719 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:13.713+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:13.721 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:13.715+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:13.851 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:13.847+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:14.095 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:14.088+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:14.202 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:14.196+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:14.604 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:14.597+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:14.693 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:14.685+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:14.706 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:14.699+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:14.707 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:14.702+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:14.730 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:14.720+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:14.808 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:14.803+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:15.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:15.096+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:15.258 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:15.252+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:15.587 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:15.580+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:15.655 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:15.649+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:15.723 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:15.716+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:15.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:15.719+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:15.748 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:15.742+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:15.817 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:15.811+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:16.125 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:16.118+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:16.281 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:16.277+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:16.565 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:16.558+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:16.641 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:16.636+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:16.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:16.737+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:16.759 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:16.754+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:16.759 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:16.756+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:16.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:16.834+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:17.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:17.160+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:17.270 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:17.264+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:17.533 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:17.527+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:17.631 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:17.626+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:17.731 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:17.724+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:17.761 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:17.752+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:17.762 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:17.753+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:17.836 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:17.831+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:18.199 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:18.192+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:18.233 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:18.227+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:18.534 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:18.528+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:18.689 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:18.679+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:18.764 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:18.756+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:18.796 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:18.790+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:18.796 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:18.793+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:18.850 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:18.845+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:19.200 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:19.195+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:19.272 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:19.265+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:19.516 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:19.510+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:19.708 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:19.702+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:19.746 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:19.741+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:19.784 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:19.780+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:19.785 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:19.781+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:19.816 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:19.811+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:20.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:20.173+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:20.250 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:20.245+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:20.564 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:20.558+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:20.696 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:20.690+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:20.779 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:20.774+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:20.780 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:20.776+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:20.799 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:20.793+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:20.852 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:20.846+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:21.180 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:21.175+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:21.281 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:21.276+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:21.597 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:21.591+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:21.701 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:21.695+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:21.735 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:21.731+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:21.739 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:21.734+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:21.832 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:21.826+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:21.845 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:21.840+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:22.192 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:22.185+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:22.317 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:22.306+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:22.612 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:22.606+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:22.687 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:22.681+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:22.718 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:22.713+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:22.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:22.716+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:22.832 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:22.826+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:22.833 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:22.826+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:23.206 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:23.199+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:23.319 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:23.314+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:23.660 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:23.653+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:23.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:23.704+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:23.731 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:23.727+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:23.733 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:23.730+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:23.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:23.869+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:23.885 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:23.879+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:24.203 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:24.197+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:24.321 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:24.316+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:24.666 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:24.660+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:24.681 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:24.675+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:24.712 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:24.706+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:24.714 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:24.709+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:24.895 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:24.890+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:24.909 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:24.903+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:25.258 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:25.252+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:25.291 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:25.287+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:25.627 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:36:25.620+0000 1694c700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.400469+0000 front 2019-11-29T16:35:07.749785+0000 (oldest deadline 2019-11-29T16:35:52.967638+0000) 2019-11-29T16:36:25.699 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:25.694+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:25.700 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:25.696+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:25.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:36:25.718+0000 1694c700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:09.346056+0000 front 2019-11-29T16:35:09.298630+0000 (oldest deadline 2019-11-29T16:35:54.582660+0000) 2019-11-29T16:36:25.878 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:25.869+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:25.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:36:25.949+0000 1694c700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.562074+0000 front 2019-11-29T16:35:08.441778+0000 (oldest deadline 2019-11-29T16:35:54.005530+0000) 2019-11-29T16:36:26.268 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:36:26.261+0000 1694c700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:08.321468+0000 front 2019-11-29T16:35:08.343679+0000 (oldest deadline 2019-11-29T16:35:52.898524+0000) 2019-11-29T16:36:26.315 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:36:26.309+0000 1694c700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:11.199668+0000 front 2019-11-29T16:35:11.230032+0000 (oldest deadline 2019-11-29T16:35:55.889555+0000) 2019-11-29T16:36:26.575 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:36:26.570+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 48sec e183 v183) 2019-11-29T16:36:26.726 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:26.720+0000 1694c700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:10.727128+0000 front 2019-11-29T16:35:10.763232+0000 (oldest deadline 2019-11-29T16:35:55.419684+0000) 2019-11-29T16:36:26.727 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:26.722+0000 1694c700 -1 osd.2 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:26.844 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:36:26.840+0000 1694c700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:35:07.812633+0000 front 2019-11-29T16:35:07.725516+0000 (oldest deadline 2019-11-29T16:35:53.588202+0000) 2019-11-29T16:36:27.706 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:27.701+0000 1694c700 -1 osd.2 184 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:36:28.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:36:28.689+0000 1694c700 -1 osd.2 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98603 2.21s0 2:86745601:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.614_%e8%94%8b%ed%88%8d%e5%98%86%e5%98%86%e1%a0%84%e1%a0%82%ed%88%8f%e9%90%8a%ed%88%8e%ef%84%91%eb%8c%8d%e2%98%83%eb%8c%8e%e5%98%88%ed%88%8e:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e183) 2019-11-29T16:37:01.195 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:01.189+0000 33a70700 -1 osd.2 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.271 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:01.255+0000 114ab700 -1 osd.2 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.531 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:01.374+0000 35a74700 -1 osd.2 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.540 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:01.375+0000 36a76700 -1 osd.2 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:01.527+0000 3aa7e700 -1 osd.3 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.623 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:01.518+0000 38279700 -1 osd.2 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:01.660 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:01.645+0000 34271700 -1 osd.3 187 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:06.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:06.399+0000 124ad700 -1 osd.3 188 build_incremental_map_msg missing incremental map 187 2019-11-29T16:37:23.785 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:37:23.763+0000 20e22700 -1 RGWWatcher::handle_error cookie 5462024448 err (107) Transport endpoint is not connected 2019-11-29T16:37:23.984 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:23.979+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:24.943 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:24.936+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:26.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:25.989+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:26.581 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:26.574+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:26.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:26.816+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:26.873 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:26.865+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:27.030 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:27.023+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:27.619 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:27.611+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:27.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:27.618+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:27.814 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:27.807+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:27.865 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:27.858+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:28.080 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:28.071+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:28.593 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:28.586+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:28.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:28.594+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:28.749 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:28.743+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:28.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:28.834+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:28.850 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:28.844+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:29.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:29.030+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:29.555 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:29.549+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:29.596 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:29.590+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:29.796 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:29.790+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:29.813 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:29.803+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:29.877 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:29.870+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:30.047 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:30.033+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:30.518 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:30.511+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:30.566 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:30.559+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:30.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:30.747+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:30.860 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:30.853+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:30.862 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:30.856+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:31.032 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:31.025+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:31.530 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:31.523+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:31.619 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:31.612+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:31.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:31.789+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:31.864 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:31.858+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:31.921 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:31.909+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:32.016 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:32.009+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:32.538 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:32.529+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:32.577 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:32.570+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:32.657 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:32.648+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:32.793 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:32.788+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:32.824 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:32.818+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:32.907 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:32.901+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:32.996 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:32.988+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:33.546 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:33.541+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:33.602 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:33.597+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:33.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:33.619+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:33.827 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:33.823+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:33.840 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:33.836+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:33.932 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:33.921+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:34.012 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:34.005+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:34.514 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:34.510+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:34.616 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:34.612+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:34.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:34.614+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:34.878 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:34.873+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:34.941 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:34.935+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:34.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:34.991+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:35.069 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:35.061+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:35.503 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:35.499+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:35.633 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:35.624+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:35.659 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:35.649+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:35.925 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:35.920+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:35.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:35.956+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:36.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:36.001+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:36.089 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:36.081+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:36.533 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:36.521+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:36.610 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:36.604+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:36.709 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:36.700+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:36.893 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:36.888+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:36.986 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:36.980+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:37.000 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:36.994+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:37.142 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:37.135+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:37.561 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:37.550+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:37.603 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:37.598+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:37.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:37.694+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:37.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:37.858+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:37.987 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:37.981+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:38.000 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:37.993+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:38.166 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:38.160+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:38.533 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:38.529+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:38.665 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:38.648+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:38.679 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:38.675+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:38.824 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:38.820+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:39.014 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:39.008+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:39.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:39.022+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:39.212 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:39.202+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:39.578 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:39.565+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:39.581 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:39.575+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:39.640 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:39.633+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:39.677 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:39.672+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:39.877 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:39.870+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:39.995 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:39.990+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:40.037 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:40.028+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:40.225 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:40.220+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:40.613 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:40.608+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:40.618 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:40.612+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:40.630 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:40.617+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:40.645 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:40.640+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:40.879 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:40.875+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:40.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:40.967+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:41.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:40.995+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:41.206 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:41.201+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:41.609 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:41.604+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:41.613 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:41.607+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:41.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:41.655+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:41.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:41.658+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:41.903 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:41.897+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:41.988 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:41.982+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:42.020 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:42.014+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:42.247 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:42.241+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:42.642 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:42.626+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:42.659 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:42.653+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:42.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:42.677+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:42.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:42.681+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:42.927 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:42.922+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:43.004 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:42.999+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:43.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:43.008+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:43.241 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:43.235+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:43.634 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:43.628+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:43.669 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:43.659+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:43.729 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:43.716+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:43.729 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:43.719+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:43.944 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:43.940+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:43.993 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:43.987+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:44.049 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:44.044+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:44.238 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:44.233+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:44.591 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:44.585+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:44.717 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:44.712+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:44.756 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:44.750+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:44.758 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:44.752+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:44.940 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:44.932+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:44.978 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:44.971+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:45.046 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:45.039+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:45.233 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:45.227+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:45.633 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:45.620+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:45.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:45.751+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:45.779 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:45.774+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:45.780 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:45.776+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:45.908 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:45.903+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:45.971 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:45.965+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:46.038 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:46.032+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:46.212 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:46.207+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:46.655 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:46.649+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:46.729 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:46.723+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:46.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:46.763+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:46.774 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:46.768+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:46.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:46.909+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:46.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:46.969+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:46.999 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:46.994+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:47.255 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:47.249+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:47.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:47.677+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:47.738 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:47.731+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:47.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:47.734+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:47.784 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:47.777+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:47.952 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:47.946+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:47.990 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:47.984+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:48.020 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:48.014+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:48.299 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:48.293+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:48.706 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:48.700+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:48.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:48.712+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:48.719 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:48.715+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:48.750 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:48.744+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:49.003 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:48.998+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:49.038 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:49.032+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:49.040 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:49.035+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:49.356 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:49.348+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:49.397 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:37:49.374+0000 20e22700 -1 RGWWatcher::handle_error cookie 363800608 err (107) Transport endpoint is not connected 2019-11-29T16:37:49.674 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:49.668+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:49.705 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:49.699+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:49.709 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:49.704+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:49.710 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:49.706+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:49.997 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:49.991+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:50.008 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:50.002+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:50.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:50.070+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:50.322 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:50.316+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:50.689 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:50.685+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:50.691 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:50.686+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:50.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:50.721+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:50.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:50.722+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:51.023 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:51.018+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:51.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:51.038+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:51.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:51.067+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:51.376 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:51.370+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:51.658 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:51.652+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:51.692 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:51.685+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:51.758 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:51.748+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:51.758 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:51.751+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:51.979 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:51.975+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:52.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:52.034+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:52.106 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:52.098+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:52.342 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:52.335+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:52.646 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:52.642+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:52.699 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:52.693+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:52.717 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:52.709+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:52.717 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:52.712+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:52.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:52.967+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:53.063 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:53.054+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:53.063 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:53.056+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:53.342 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:53.333+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:53.662 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:53.657+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:53.665 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:53.659+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:53.740 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:53.735+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:53.743 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:53.740+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:54.001 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:53.996+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:54.002 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:53.997+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:54.052 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:54.046+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:54.107 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:54.102+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:54.338 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:54.332+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:54.687 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:54.682+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:54.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:54.701+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:54.708 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:54.703+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:54.714 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:54.709+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:55.006 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:55.000+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:55.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:55.005+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:55.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:55.010+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:55.149 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:55.143+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:55.370 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:55.362+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:55.701 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:55.696+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:55.704 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:55.700+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:55.707 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:55.701+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:55.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:55.722+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:56.025 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:56.018+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:56.030 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:56.025+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:56.030 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:56.027+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:56.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:56.185+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:56.400 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:56.393+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:56.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:56.681+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:56.689 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:56.684+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:56.754 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:56.747+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:56.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:56.745+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:56.865 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:37:56.855+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e188 v188) 2019-11-29T16:37:57.070 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:57.064+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:57.076 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:57.073+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:57.078 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:57.075+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:57.194 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:57.188+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:57.452 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:57.447+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:57.701 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:57.684+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:57.701 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:57.688+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:57.709 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:57.703+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:57.794 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:57.790+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:58.101 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:58.095+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:58.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:58.100+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:58.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:58.102+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:58.210 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:58.204+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:58.476 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:58.468+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:58.720 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:58.714+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:58.723 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:58.718+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:58.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:58.752+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:58.842 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:58.836+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:37:59.061 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:37:59.054+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:37:59.132 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:59.126+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:37:59.133 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:37:59.128+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:59.200 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:59.193+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:37:59.201 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:37:59.196+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:37:59.445 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:37:59.439+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:37:59.766 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:59.760+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:37:59.767 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:37:59.762+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:37:59.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:37:59.798+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:37:59.852 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:37:59.845+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:00.090 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:00.084+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:00.177 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:00.171+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:00.178 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:00.173+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:00.186 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:00.179+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:00.186 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:00.181+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:00.493 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:00.487+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:00.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:00.766+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:00.774 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:00.768+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:00.791 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:00.771+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:00.872 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:00.867+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:01.079 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:01.073+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:01.208 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:01.201+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:01.210 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:01.204+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:01.221 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:01.215+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:01.222 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:01.217+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:01.478 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:01.471+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:01.781 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:01.766+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:01.819 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:01.812+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:01.819 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:01.813+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:01.861 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:01.851+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:01.876 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:38:01.867+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e188 v188) 2019-11-29T16:38:02.076 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:02.070+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:02.173 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:02.167+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:02.174 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:02.169+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:02.254 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:02.248+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:02.257 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:02.252+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:02.446 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:02.439+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:02.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:02.753+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:02.790 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:02.783+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:02.792 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:02.786+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:02.838 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:02.832+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:03.082 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:03.075+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:03.209 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:03.202+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:03.209 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:03.204+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:03.311 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:03.304+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:03.311 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:03.306+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:03.455 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:03.448+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:03.752 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:03.745+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:03.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:03.788+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:03.796 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:03.790+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:03.858 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:03.852+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:04.128 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:04.121+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:04.182 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:04.175+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:04.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:04.180+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:04.310 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:04.303+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:04.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:04.308+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:04.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:04.418+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:04.744 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:04.737+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:04.829 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:04.816+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:04.829 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:04.819+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:04.844 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:04.830+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:05.145 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:05.138+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:05.177 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:05.169+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:05.180 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:05.175+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:05.341 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:05.336+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:05.344 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:05.339+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:05.477 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:05.470+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:05.754 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:05.745+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:05.807 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:05.799+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:05.808 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:05.803+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:05.854 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:05.848+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:06.101 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:06.095+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:06.139 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:06.132+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:06.140 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:06.135+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:06.372 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:06.365+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:06.373 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:06.367+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:06.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:38:06.517+0000 1694c700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.289328+0000 front 2019-11-29T16:36:43.265313+0000 (oldest deadline 2019-11-29T16:37:26.623048+0000) 2019-11-29T16:38:06.735 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:06.728+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:06.806 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:06.800+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:06.811 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:06.806+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:06.819 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:06.814+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:06.891 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:38:06.884+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e188 v188) 2019-11-29T16:38:07.080 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:07.074+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:07.179 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:07.173+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:07.181 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:07.174+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:07.367 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:07.360+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:07.368 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:07.363+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:07.744 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:07.736+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:07.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:07.808+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:07.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:07.809+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:07.856 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:07.851+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:08.043 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:08.036+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:08.229 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:08.221+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:08.229 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:08.224+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:08.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:08.342+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:08.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:08.345+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:08.726 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:08.719+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:08.833 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:08.828+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:08.836 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:08.832+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:08.890 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:08.884+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:09.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:09.033+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:09.255 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:09.248+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:09.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:09.250+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:09.345 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:09.338+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:09.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:09.341+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:09.765 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:09.756+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:09.804 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:09.798+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:09.809 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:09.803+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:09.859 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:09.852+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:10.067 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:10.060+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:10.293 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:10.286+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:10.293 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:10.289+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:10.308 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:10.303+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:10.309 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:10.304+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:10.769 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:10.764+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:10.853 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:10.846+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:10.859 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:10.852+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:10.911 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:10.904+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:11.067 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:38:11.061+0000 1694c700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:43.385689+0000 front 2019-11-29T16:36:43.407231+0000 (oldest deadline 2019-11-29T16:37:23.866700+0000) 2019-11-29T16:38:11.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:11.306+0000 1694c700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.464610+0000 front 2019-11-29T16:36:42.508946+0000 (oldest deadline 2019-11-29T16:37:28.277006+0000) 2019-11-29T16:38:11.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:11.308+0000 1694c700 -1 osd.3 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:11.329 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:11.323+0000 1694c700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.488929+0000 front 2019-11-29T16:36:40.491325+0000 (oldest deadline 2019-11-29T16:37:26.216093+0000) 2019-11-29T16:38:11.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:11.325+0000 1694c700 -1 osd.6 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:11.799 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:38:11.793+0000 1694c700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:42.530729+0000 front 2019-11-29T16:36:42.533052+0000 (oldest deadline 2019-11-29T16:37:26.678425+0000) 2019-11-29T16:38:11.855 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:11.849+0000 1694c700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:36:59.433898+0000 (oldest deadline 2019-11-29T16:37:39.433898+0000) 2019-11-29T16:38:11.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:38:11.851+0000 1694c700 -1 osd.0 188 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-11-29T16:38:11.937 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:38:11.926+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e188 v188) 2019-11-29T16:38:11.953 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:38:11.942+0000 1694c700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:36:40.349962+0000 front 2019-11-29T16:36:40.341130+0000 (oldest deadline 2019-11-29T16:37:26.114040+0000) 2019-11-29T16:38:12.568 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:12.522+0000 1694c700 -1 osd.6 189 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:12.680 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:12.627+0000 1694c700 -1 osd.3 189 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:13.645 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:38:13.550+0000 1694c700 -1 osd.6 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98880 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:13.665 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:38:13.654+0000 1694c700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:98874 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363800608] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-11-29T16:38:33.112 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:38:33.050+0000 1694c700 -1 osd.1 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:98903 6.1 6.c9be1e91 (undecoded) ondisk+read+known_if_redirected e188) 2019-11-29T16:39:17.256 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:26:30.145 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.264 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:26:30.153 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.268 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:26:30.157 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.275 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:26:30.163 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.286 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:26:30.060 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.298 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:26:30.073 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.517 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:26:30.405 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.519 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:26:30.408 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:39:17.640 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:26:30.414 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:00.613 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:00.609+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:00.652 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:40:00.648+0000 1694c700 -1 osd.2 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.463932+0000 front 2019-11-29T16:39:17.208559+0000 (oldest deadline 2019-11-29T16:40:00.431833+0000) 2019-11-29T16:40:01.269 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:01.263+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:01.370 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:01.365+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:01.473 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:01.465+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:01.597 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:01.591+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:01.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:40:01.680+0000 1694c700 -1 osd.2 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.463932+0000 front 2019-11-29T16:39:17.208559+0000 (oldest deadline 2019-11-29T16:40:00.431833+0000) 2019-11-29T16:40:01.703 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:01.697+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:02.309 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:02.291+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:02.349 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:02.344+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:02.384 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:02.377+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:02.442 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:02.434+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:02.587 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:02.581+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:02.751 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:02.744+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:03.292 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:03.285+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:03.353 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:03.346+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:03.381 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:03.375+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:03.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:03.410+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:03.553 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:03.548+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:03.716 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:03.708+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:04.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:04.275+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:04.337 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:04.331+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:04.350 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:04.345+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:04.377 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:04.368+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:04.528 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:04.521+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:04.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:04.748+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:05.296 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:05.290+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:05.343 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:05.336+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:05.360 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:05.354+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:05.381 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:05.376+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:05.550 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:05.544+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:05.733 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:05.727+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:06.303 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:06.297+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:06.324 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:06.318+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:06.330 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:06.323+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:06.436 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:06.430+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:06.533 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:06.529+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:06.741 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:06.735+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:07.313 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:07.307+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:07.336 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:07.329+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:07.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:07.366+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:07.453 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:27:20.342 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.467 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:27:20.357 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.469 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:27:20.357 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.487 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:07.483+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:07.499 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:27:20.387 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.508 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:07.504+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:07.512 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:27:20.287 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.521 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:27:20.296 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.529 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:27:20.418 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.540 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:27:20.429 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.604 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:27:20.379 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:40:07.740 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:07.735+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:08.311 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:08.305+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:08.338 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:08.332+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:08.349 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:08.341+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:08.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:08.476+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:08.516 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:08.511+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:08.735 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:08.729+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:09.296 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:09.290+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:09.345 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:09.338+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:09.400 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:09.394+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:09.451 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:09.447+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:09.551 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:09.545+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:09.789 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:09.782+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:10.282 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:10.276+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:10.366 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:10.360+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:10.407 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:10.402+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:10.442 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:10.435+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:10.553 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:10.548+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:10.768 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:10.762+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:11.265 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:11.259+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:11.391 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:11.384+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:11.420 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:11.413+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:11.451 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:11.445+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:11.511 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:11.505+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:11.810 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:11.804+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:12.294 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:12.288+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:12.368 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:12.363+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:12.418 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:12.411+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:12.509 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:12.502+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:12.522 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:12.518+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:12.818 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:12.813+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:13.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:13.249+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:13.419 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:13.414+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:13.422 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:13.415+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:13.502 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:13.496+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:13.556 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:13.550+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:13.783 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:13.777+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:14.294 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:14.287+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:14.373 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:14.369+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:14.459 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:14.454+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:14.523 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:14.519+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:14.542 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:14.537+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:14.827 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:14.823+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:15.302 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:15.296+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:15.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:15.379+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:15.479 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:15.472+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:15.544 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:15.538+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:15.570 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:15.565+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:15.814 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:15.802+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:16.264 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:16.258+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:16.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:16.356+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:16.514 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:16.507+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:16.552 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:16.547+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:16.619 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:16.613+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:16.806 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:16.800+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:17.307 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:17.300+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:17.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:17.396+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:17.533 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:17.527+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:17.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:17.548+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:17.633 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:17.626+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:17.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:17.842+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:18.281 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:18.275+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:18.435 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:18.430+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:18.490 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:18.484+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:18.547 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:18.541+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:18.611 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:18.604+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:18.896 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:18.891+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:19.294 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:19.288+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:19.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:19.419+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:19.482 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:19.476+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:19.523 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:19.518+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:19.634 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:19.629+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:19.930 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:19.924+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:20.276 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:20.271+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:20.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:20.377+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:20.512 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:20.506+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:20.572 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:20.565+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:20.594 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:20.588+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:20.901 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:20.895+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:21.281 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:21.275+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:21.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:21.380+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:21.550 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:21.543+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:21.554 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:21.547+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:21.657 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:21.647+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:21.919 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:21.913+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:22.258 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:22.252+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:22.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:22.420+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:22.554 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:22.548+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:22.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:22.595+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:22.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:22.696+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:22.942 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:22.936+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:23.229 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:23.223+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:23.450 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:23.444+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:23.558 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:23.553+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:23.655 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:23.649+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:23.663 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:23.656+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:23.951 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:23.945+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:24.234 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:24.228+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:24.461 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:24.455+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:24.578 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:24.571+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:24.665 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:24.660+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:24.669 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:24.663+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:24.972 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:24.963+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:25.222 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:25.216+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:25.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:25.489+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:25.594 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:25.588+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:25.697 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:25.691+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:25.697 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:25.691+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:25.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:25.934+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:26.261 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:26.250+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:26.484 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:26.478+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:26.562 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:26.556+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:26.688 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:26.683+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:26.700 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:26.695+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:26.993 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:26.987+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:27.264 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:27.258+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:27.521 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:27.515+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:27.597 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:27.592+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:27.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:27.700+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:27.743 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:27.737+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:28.024 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:28.018+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:28.246 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:28.239+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:28.508 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:28.500+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:28.643 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:28.635+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:28.717 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:28.710+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:28.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:28.754+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:29.071 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:29.060+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:29.249 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:29.243+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:29.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:29.532+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:29.684 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:29.676+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:29.702 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:29.696+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:29.801 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:29.794+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:30.031 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:30.025+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:30.243 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:30.237+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:30.532 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:30.527+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:30.693 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:30.686+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:30.694 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:30.687+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:30.841 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:30.836+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:31.009 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:31.003+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:31.215 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:31.210+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:31.553 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:31.547+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:31.665 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:31.659+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:31.740 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:31.734+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:31.884 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:31.878+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:32.031 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:32.025+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:32.264 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:32.257+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:32.465 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:40:32.460+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e194 v194) 2019-11-29T16:40:32.578 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:32.571+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:32.663 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:32.656+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:32.763 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:32.757+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:32.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:32.848+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:33.017 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:33.010+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:33.283 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:33.276+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:33.613 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:33.607+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:33.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:33.644+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:33.787 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:33.782+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:33.826 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:33.819+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:33.998 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:33.991+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:34.322 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:34.316+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:34.636 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:34.629+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:34.658 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:34.653+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:34.818 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:34.814+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:34.861 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:34.855+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:35.048 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:35.043+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:35.368 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:35.360+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:35.615 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:35.608+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:35.646 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:35.641+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:35.837 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:35.833+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:35.856 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:35.849+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:36.028 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:36.023+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:36.332 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:36.325+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:36.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:36.594+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:36.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:36.660+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:40:36.817 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:40:36.812+0000 1694c700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:16.841700+0000 front 2019-11-29T16:39:17.387116+0000 (oldest deadline 2019-11-29T16:40:00.661440+0000) 2019-11-29T16:40:36.897 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:40:36.890+0000 1694c700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.909573+0000 front 2019-11-29T16:39:19.858907+0000 (oldest deadline 2019-11-29T16:40:00.730073+0000) 2019-11-29T16:40:37.022 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:40:37.015+0000 1694c700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:15.635245+0000 front 2019-11-29T16:39:15.877537+0000 (oldest deadline 2019-11-29T16:40:01.489934+0000) 2019-11-29T16:40:37.372 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:40:37.365+0000 1694c700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:17.418616+0000 front 2019-11-29T16:39:17.396938+0000 (oldest deadline 2019-11-29T16:40:01.470006+0000) 2019-11-29T16:40:37.536 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:40:37.527+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e194 v194) 2019-11-29T16:40:37.575 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:40:37.567+0000 1694c700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:19.867118+0000 front 2019-11-29T16:39:19.662841+0000 (oldest deadline 2019-11-29T16:40:00.693888+0000) 2019-11-29T16:40:37.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:40:37.696+0000 1694c700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:39:18.151400+0000 front 2019-11-29T16:39:18.261637+0000 (oldest deadline 2019-11-29T16:39:59.852551+0000) 2019-11-29T16:41:10.392 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.379+0000 39a7c700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.683 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.639+0000 34a72700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.653+0000 38a7a700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.750 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.653+0000 3927b700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.868 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.659+0000 3a27d700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.921 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.679+0000 3927b700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:10.988 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.736+0000 39a7c700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:11.150 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.935+0000 3b27f700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:11.216 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:10.867+0000 3927b700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:11.295 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:11.012+0000 3a27d700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:11.319 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:11.107+0000 3a27d700 -1 osd.0 199 build_incremental_map_msg missing incremental map 199 2019-11-29T16:41:35.624 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:35.618+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:36.000 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:35.993+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:36.589 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:36.584+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:36.780 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:36.772+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:36.996 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:36.990+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:37.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:37.608+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:37.803 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:41:37.795+0000 20e22700 -1 RGWWatcher::handle_error cookie 821003888 err (107) Transport endpoint is not connected 2019-11-29T16:41:37.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:37.802+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:38.000 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:37.995+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:38.241 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:38.236+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:38.407 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:38.403+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:38.577 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:38.570+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:38.821 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:38.815+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:39.013 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:39.007+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:39.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:39.199+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:39.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:39.360+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:39.414 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:39.408+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:39.544 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:39.538+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:39.645 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:39.639+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:39.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:39.843+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:39.992 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:39.986+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:40.162 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:40.157+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:40.416 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:40.410+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:40.420 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:40.415+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:40.588 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:40.583+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:40.674 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:40.668+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:40.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:40.844+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:41.002 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:40.996+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:41.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:41.177+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:41.410 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:41.406+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:41.434 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:41.429+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:41.572 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:41.567+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:41.707 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:41.700+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:41.872 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:41.866+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:41.997 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:41.984+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:42.141 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:42.137+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:42.404 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:42.400+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:42.413 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:42.408+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:42.598 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:42.592+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:42.727 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:42.721+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:42.874 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:42.869+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:43.034 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:43.029+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:43.181 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:43.177+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:43.403 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:43.399+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:43.419 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:43.411+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:43.604 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:43.598+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:43.729 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:43.722+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:43.903 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:43.898+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:43.998 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:43.991+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:44.152 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:44.147+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:44.415 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:44.407+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:44.469 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:44.464+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:44.584 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:44.577+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:44.745 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:44.739+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:44.858 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:44.853+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:44.984 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:44.978+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:45.194 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:45.189+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:45.431 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:45.426+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:45.454 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:45.449+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:45.571 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:45.565+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:45.726 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:45.721+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:45.817 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:45.812+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:45.987 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:45.982+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:46.173 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:46.168+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:46.471 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:46.466+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:46.507 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:46.501+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:46.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:46.530+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:46.713 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:46.707+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:46.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:46.798+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:47.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:47.020+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:47.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:47.200+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:47.490 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:47.485+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:47.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:47.488+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:47.496 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:47.491+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:47.546 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:47.539+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:47.730 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:47.724+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:47.813 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:47.806+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:48.058 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:48.052+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:48.202 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:48.195+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:48.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:48.460+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:48.469 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:48.463+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:48.524 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:48.518+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:48.592 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:48.586+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:48.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:48.759+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:48.850 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:48.843+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:49.029 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:49.022+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:49.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:49.206+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:49.448 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:49.441+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:49.450 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:49.444+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:49.489 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:49.482+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:49.640 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:49.635+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:49.790 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:49.783+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:49.841 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:49.834+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:50.022 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:50.015+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:50.239 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:50.234+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:50.424 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:50.417+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:50.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:50.420+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:50.447 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:50.441+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:50.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:50.625+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:50.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:50.792+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:50.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:50.818+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:51.059 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:51.052+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:51.215 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:51.209+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:51.419 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:51.412+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:51.420 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:51.414+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:51.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:51.475+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:51.626 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:51.621+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:51.826 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:51.819+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:51.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:51.842+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:52.081 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:52.074+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:52.190 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:52.183+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:52.455 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:52.448+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:52.458 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:52.452+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:52.459 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:52.454+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:52.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:52.593+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:52.830 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:52.824+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:52.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:52.870+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:53.087 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:53.081+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:53.176 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:53.169+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:53.450 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:53.444+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:53.451 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:53.443+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:53.452 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:53.446+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:53.621 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:53.615+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:53.788 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:53.781+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:53.928 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:53.921+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:54.140 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:54.133+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:54.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:54.179+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:54.408 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:54.401+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:54.432 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:54.426+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:54.436 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:54.431+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:54.626 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:54.620+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:54.800 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:54.794+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:54.949 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:54.944+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:55.155 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:55.149+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:55.166 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:55.160+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:55.365 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:55.360+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:55.451 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:55.444+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:55.455 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:55.450+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:55.617 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:55.610+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:55.811 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:55.804+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:56.002 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:55.995+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:56.159 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:56.153+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:56.209 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:56.196+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:56.362 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:56.355+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:56.421 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:56.411+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:56.421 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:56.415+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:56.623 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:56.616+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:56.777 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:56.770+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:57.003 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:56.996+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:57.198 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:57.185+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:57.198 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:57.192+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:57.397 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:57.391+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:57.469 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:57.461+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:57.470 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:57.465+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:57.582 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:57.576+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:57.732 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:57.726+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:58.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:58.009+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:58.213 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:58.206+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:58.226 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:58.221+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:58.382 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:58.375+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:58.428 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:58.420+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:58.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:58.422+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:58.602 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:58.596+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:58.777 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:58.770+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:58.992 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:58.987+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:41:59.197 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:41:59.192+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:41:59.257 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:41:59.251+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:41:59.386 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:41:59.380+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:41:59.465 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:59.458+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:41:59.467 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:41:59.461+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:41:59.599 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:41:59.593+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:41:59.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:41:59.741+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:41:59.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:41:59.945+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:00.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:00.188+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:00.225 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:00.219+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:00.344 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:00.337+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:00.477 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:00.471+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:00.478 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:00.473+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:00.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:00.581+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:00.707 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:00.701+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:00.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:00.911+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:01.217 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:01.211+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:01.224 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:01.218+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:01.350 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:01.344+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:01.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:01.487+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:01.496 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:01.490+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:01.630 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:01.624+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:01.729 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:01.723+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:01.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:01.951+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:02.227 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:02.221+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:02.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:02.236+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:02.359 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:02.353+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:02.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:02.462+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:02.471 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:02.466+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:02.678 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:02.671+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:02.751 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:02.744+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:02.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:02.956+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:03.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:03.195+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:03.256 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:03.250+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:03.395 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:03.389+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:03.489 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:03.480+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:03.491 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:03.484+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:03.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:03.722+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:03.765 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:03.759+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:03.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:03.989+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:04.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:04.226+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:04.267 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:04.261+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:04.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:04.419+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:04.453 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:04.447+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:04.457 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:04.452+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:04.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:04.745+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:04.757 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:04.751+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:05.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:04.995+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:05.272 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:05.267+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:05.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:05.302+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:05.433 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:05.427+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:05.462 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:05.457+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:05.466 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:05.462+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:05.766 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:05.760+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:05.782 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:05.777+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:05.933 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:42:05.876+0000 20e22700 -1 RGWWatcher::handle_error cookie 5293352352 err (107) Transport endpoint is not connected 2019-11-29T16:42:05.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:05.988+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:06.263 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:06.259+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:06.297 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:06.291+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:06.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:06.477+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:06.504 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:06.499+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:06.505 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:06.501+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:06.734 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:06.728+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:06.778 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:06.773+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:07.006 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:06.999+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:07.274 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:07.270+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:07.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:07.318+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:07.500 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:07.495+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:07.501 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:07.497+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:07.514 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:07.509+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:07.749 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:07.744+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:07.776 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:07.770+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:07.786 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:42:07.781+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e199 v199) 2019-11-29T16:42:08.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:08.011+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:08.261 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:08.256+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:08.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:08.373+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:08.493 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:08.488+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:08.494 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:08.490+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:08.527 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:08.521+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:08.736 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:08.730+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:08.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:08.732+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:09.027 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:09.020+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:09.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:09.308+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:09.432 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:09.427+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:09.488 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:09.482+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:09.491 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:09.486+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:09.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:09.491+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:09.757 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:09.752+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:09.780 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:09.774+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:09.986 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:09.980+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:10.275 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:10.269+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:10.276 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:10.271+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:10.438 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:10.432+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:10.455 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:10.449+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:10.550 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:10.542+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:10.551 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:10.546+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:10.747 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:10.741+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:10.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:10.740+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:11.027 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:11.020+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:11.311 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:11.306+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:11.312 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:11.309+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:11.414 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:11.408+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:11.508 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:11.498+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:11.582 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:11.576+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:11.586 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:11.581+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:11.734 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:11.729+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:11.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:11.781+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:12.067 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:12.063+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:12.361 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:12.356+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:12.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:12.358+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:12.451 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:12.445+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:12.554 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:12.547+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:12.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:12.568+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:12.578 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:12.572+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:12.760 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:12.755+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:12.809 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:42:12.800+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e199 v199) 2019-11-29T16:42:12.819 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:12.814+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:13.102 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:13.096+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:13.409 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:13.403+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:13.414 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:13.408+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:13.415 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:13.411+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:13.530 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:13.521+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:13.560 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:13.555+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:13.565 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:13.561+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:13.800 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:13.794+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:13.814 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:13.807+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:14.058 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:14.052+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:14.366 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:14.362+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:14.366 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:14.363+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:14.444 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:14.438+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:14.570 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:14.564+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:14.597 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:14.590+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:14.598 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:14.593+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:14.828 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:14.824+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:14.831 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:14.825+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:15.109 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:15.103+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:15.406 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:15.400+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:15.407 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:15.402+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:15.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:15.438+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:15.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:15.601+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:15.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:15.605+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:15.610 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:15.604+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:15.786 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:15.782+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:15.826 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:15.820+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:16.083 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:16.077+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:16.409 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:16.404+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:16.444 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:16.439+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:16.447 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:16.443+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:16.610 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:16.605+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:16.614 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:16.610+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:16.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:16.609+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:16.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:16.757+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:16.800 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:16.784+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:17.086 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:17.079+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:17.382 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:17.376+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:17.453 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:17.447+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:17.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:17.450+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:17.644 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:17.638+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:17.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:17.651+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:17.659 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:17.655+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:17.769 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:17.764+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:17.803 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:17.798+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:17.820 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:42:17.814+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e199 v199) 2019-11-29T16:42:18.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:18.038+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:18.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:18.423+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:18.435 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:18.430+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:18.438 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:18.434+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:18.638 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:18.632+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:18.679 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:18.673+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:18.680 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:18.676+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:18.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:18.744+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:18.773 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:18.768+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:18.999 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:18.994+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:19.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:19.437+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:19.460 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:19.456+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:19.462 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:19.458+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:19.640 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:19.633+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:19.641 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:19.636+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:19.679 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:19.673+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:19.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:19.742+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:19.802 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:19.797+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:19.992 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:19.986+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:20.484 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:20.479+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:20.485 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:20.481+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:20.488 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:20.483+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:20.603 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:20.598+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:20.608 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:20.603+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:20.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:20.726+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:20.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:20.744+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:20.853 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:20.847+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:20.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:20.951+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:21.452 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:21.446+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:21.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:21.475+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:21.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:21.478+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:21.626 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:21.620+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:21.627 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:21.622+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:21.737 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:21.731+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:21.769 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:21.763+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:21.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:21.879+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:21.923 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:21.917+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:22.489 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:42:22.483+0000 1694c700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:50.748789+0000 front 2019-11-29T16:40:50.883381+0000 (oldest deadline 2019-11-29T16:41:35.436237+0000) 2019-11-29T16:42:22.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:22.488+0000 1694c700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:52.281879+0000 front 2019-11-29T16:40:52.274275+0000 (oldest deadline 2019-11-29T16:41:38.152197+0000) 2019-11-29T16:42:22.493 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:22.489+0000 1694c700 -1 osd.3 199 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:22.673 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:22.669+0000 1694c700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:41:06.826887+0000 (oldest deadline 2019-11-29T16:41:46.826887+0000) 2019-11-29T16:42:22.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:22.670+0000 1694c700 -1 osd.0 199 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e198) 2019-11-29T16:42:22.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:42:22.743+0000 1694c700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.106374+0000 front 2019-11-29T16:40:53.865687+0000 (oldest deadline 2019-11-29T16:41:36.226934+0000) 2019-11-29T16:42:22.750 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:42:22.743+0000 1694c700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.138707+0000 front 2019-11-29T16:40:54.117564+0000 (oldest deadline 2019-11-29T16:41:35.059025+0000) 2019-11-29T16:42:22.840 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:42:22.834+0000 1694c700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.176277+0000 front 2019-11-29T16:40:54.095944+0000 (oldest deadline 2019-11-29T16:41:37.577120+0000) 2019-11-29T16:42:22.865 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:42:22.861+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e199 v199) 2019-11-29T16:42:22.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:42:22.958+0000 1694c700 -1 osd.5 199 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:40:54.734321+0000 front 2019-11-29T16:40:54.761865+0000 (oldest deadline 2019-11-29T16:41:39.336873+0000) 2019-11-29T16:42:23.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:23.637+0000 1694c700 -1 osd.3 200 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:42:23.706 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:23.643+0000 38279700 -1 osd.0 200 build_incremental_map_msg missing incremental map 199 2019-11-29T16:42:23.903 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:42:23.810+0000 1694c700 -1 osd.0 200 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99281 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e198) 2019-11-29T16:42:24.686 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:42:24.667+0000 1694c700 -1 osd.3 201 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:99326 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5293352352] snapc 0=[] ondisk+write+known_if_redirected e199) 2019-11-29T16:43:15.229 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:30:28.118 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.235 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:30:28.125 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.244 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:30:28.132 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.247 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:30:28.135 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.263 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:30:28.038 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.276 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:30:28.051 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.430 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:30:28.319 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.458 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:30:28.347 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:15.557 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:30:28.332 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:43:16.939 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:43:16.540+0000 1694c700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=23 ondisk+retry+read+known_if_redirected e203) 2019-11-29T16:43:17.818 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:43:17.608+0000 1694c700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=23 ondisk+retry+read+known_if_redirected e203) 2019-11-29T16:44:09.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:09.620+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:09.898 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:09.891+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:10.434 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:10.428+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:10.591 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:10.585+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:10.912 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:10.905+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:11.043 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:11.035+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:11.084 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:11.079+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:11.203 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:11.196+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:11.481 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:11.476+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:11.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:11.594+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:11.875 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:11.869+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:12.018 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:12.012+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:12.115 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:12.107+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:12.182 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:12.176+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:12.521 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:12.507+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:12.649 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:12.643+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:12.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:12.876+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:13.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:13.050+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:13.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:13.094+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:13.176 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:13.171+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:13.565 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:13.559+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:13.685 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:13.679+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:13.934 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:13.929+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:14.020 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:14.015+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:14.141 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:14.134+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:14.148 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:14.142+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:14.596 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:14.590+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:14.663 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:14.657+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:14.940 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:14.935+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:15.010 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:15.003+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:15.174 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:15.168+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:15.175 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:15.168+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:15.619 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:15.614+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:15.639 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:15.633+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:15.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:15.979+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:16.047 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:16.040+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:16.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:16.137+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:16.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:16.184+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:16.585 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:16.579+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:16.675 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:16.669+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:16.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:16.979+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:17.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:17.066+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:17.108 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:17.103+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:17.227 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:17.221+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:17.587 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:17.581+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:17.676 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:17.670+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:18.028 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:18.020+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:18.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:18.055+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:18.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:18.069+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:18.228 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:18.220+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:18.579 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:18.573+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:18.693 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:18.688+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:18.989 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:18.983+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:19.036 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:19.029+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:19.052 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:19.046+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:19.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:19.266+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:19.613 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:19.606+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:19.703 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:19.697+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:19.999 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:19.993+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:20.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:20.012+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:20.065 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:20.060+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:20.271 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:20.264+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:20.568 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:20.561+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:20.668 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:20.661+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:20.974 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:20.964+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:21.031 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:21.023+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:21.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:21.061+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:21.274 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:21.267+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:21.543 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:21.537+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:21.652 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:21.644+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:22.005 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:22.000+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:22.006 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:21.999+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:22.038 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:22.033+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:22.254 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:22.247+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:22.535 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:22.529+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:22.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:22.682+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:22.993 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:22.988+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:23.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:23.007+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:23.023 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:23.016+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:23.254 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:23.248+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:23.564 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:23.558+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:23.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:23.736+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:23.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:23.961+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:23.985 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:23.981+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:24.043 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:24.036+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:24.209 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:24.202+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:24.600 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:24.596+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:24.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:24.793+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:24.976 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:24.969+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:25.018 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:25.013+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:25.039 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:25.032+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:25.223 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:25.216+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:25.571 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:25.567+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:25.774 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:25.767+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:25.955 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:25.946+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:26.026 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:26.022+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:26.073 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:26.066+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:26.254 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:26.248+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:26.582 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:26.578+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:26.752 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:26.745+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:26.953 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:26.947+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:26.993 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:26.982+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:27.127 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:27.117+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:27.227 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:27.220+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:27.629 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:27.622+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:27.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:27.792+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:27.934 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:27.928+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:28.006 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:28.002+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:28.163 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:28.155+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:28.222 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:28.215+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:28.681 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:28.675+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:28.849 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:28.843+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:28.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:28.897+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:29.007 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:29.001+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:29.178 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:29.171+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:29.181 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:29.175+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:29.685 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:29.679+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:29.835 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:29.829+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:29.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:29.889+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:30.004 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:29.998+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:30.221 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:30.215+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:30.222 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:30.215+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:30.699 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:30.694+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:30.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:30.868+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:30.916 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:30.911+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:31.026 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:31.021+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:31.223 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:31.216+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:31.223 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:31.218+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:31.680 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:31.675+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:31.897 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:31.891+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:31.920 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:31.915+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:32.077 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:32.072+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:32.197 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:32.191+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:32.273 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:32.267+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:32.724 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:32.719+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:32.933 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:32.927+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:32.942 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:32.936+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:33.119 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:33.114+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:33.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:33.160+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:33.305 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:33.300+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:33.769 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:33.763+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:33.912 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:33.906+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:33.940 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:33.936+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:34.130 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:34.125+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:34.205 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:34.200+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:34.288 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:34.282+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:34.775 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:34.770+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:34.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:34.931+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:34.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:34.941+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:35.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:35.177+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:35.250 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:35.244+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:35.254 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:35.248+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:35.753 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:35.747+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:35.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:35.969+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:35.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:35.971+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:36.163 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:36.157+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:36.295 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:36.288+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:36.295 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:36.290+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:36.773 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:36.768+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:36.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:36.955+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:37.008 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:37.000+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:37.184 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:37.179+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:37.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:37.268+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:37.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:37.319+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:37.818 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:37.812+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:37.917 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:37.912+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:37.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:37.987+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:38.208 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:38.203+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:38.277 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:38.271+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:38.340 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:38.331+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:38.864 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:38.857+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:38.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:38.960+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:39.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:39.025+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:39.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:39.237+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:39.265 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:39.259+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:39.360 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:39.353+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:39.850 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:39.845+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:39.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:39.959+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:40.065 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:40.060+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:40.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:40.237+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:40.287 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:40.281+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:40.350 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:40.344+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:40.822 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:40.816+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:40.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:40.970+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:41.089 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:41.083+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:41.250 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:41.245+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:41.310 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:41.304+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:41.396 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:41.390+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:41.847 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:41.840+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:41.934 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:41.929+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:42.092 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:42.087+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:42.245 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:42.241+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:42.279 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:42.273+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:42.351 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:42.346+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:42.848 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:44:42.842+0000 1694c700 -1 osd.1 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.841891+0000 front 2019-11-29T16:43:25.020297+0000 (oldest deadline 2019-11-29T16:44:09.964295+0000) 2019-11-29T16:44:42.909 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:44:42.904+0000 1694c700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:25.997465+0000 front 2019-11-29T16:43:26.001566+0000 (oldest deadline 2019-11-29T16:44:09.363029+0000) 2019-11-29T16:44:43.066 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:44:43.060+0000 1694c700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.611983+0000 front 2019-11-29T16:43:27.503926+0000 (oldest deadline 2019-11-29T16:44:42.474220+0000) 2019-11-29T16:44:43.125 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:44:43.119+0000 1694c700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.500784+0000 front 2019-11-29T16:43:27.379930+0000 (oldest deadline 2019-11-29T16:44:09.061420+0000) 2019-11-29T16:44:43.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:44:43.219+0000 1694c700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:27.631327+0000 front 2019-11-29T16:43:27.617975+0000 (oldest deadline 2019-11-29T16:44:10.512615+0000) 2019-11-29T16:44:43.300 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:44:43.295+0000 1694c700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.025054+0000 front 2019-11-29T16:43:25.842183+0000 (oldest deadline 2019-11-29T16:44:11.089810+0000) 2019-11-29T16:44:43.358 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:44:43.352+0000 1694c700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:43:26.205657+0000 front 2019-11-29T16:43:26.203730+0000 (oldest deadline 2019-11-29T16:44:10.175673+0000) 2019-11-29T16:44:43.423 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:44:43.417+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e204 v204) 2019-11-29T16:45:41.193 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:41.182+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:42.176 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:42.169+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:42.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:42.882+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:43.206 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:43.198+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:43.580 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:43.573+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:43.899 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:43.891+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:43.907 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:43.901+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:44.244 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:44.236+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:44.485 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:44.479+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:44.488 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:44.484+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:44.578 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:44.571+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:44.682 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:44.677+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:44.892 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:44.886+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:44.936 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:44.931+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:45.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:45.244+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:45.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:45.468+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:45.475 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:45.469+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:45.545 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:45.538+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:45.718 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:45.712+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:45.811 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:45:45.806+0000 20e22700 -1 RGWWatcher::handle_error cookie 5441811792 err (107) Transport endpoint is not connected 2019-11-29T16:45:45.921 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:45.914+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:45.982 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:45.977+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:46.272 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:46.266+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:46.508 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:46.502+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:46.509 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:46.504+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:46.592 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:46.584+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:46.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:46.739+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:46.923 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:46.916+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:46.942 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:46.937+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:47.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:47.291+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:47.504 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:47.499+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:47.512 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:47.502+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:47.582 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:47.576+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:47.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:47.741+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:47.763 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:47.757+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:47.934 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:47.930+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:47.967 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:47.960+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:48.335 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:48.328+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:48.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:48.553+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:48.561 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:48.556+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:48.632 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:48.627+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:48.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:48.704+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:48.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:48.722+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:48.811 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:33:01.700 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.824 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:33:01.712 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.870 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:33:01.758 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.875 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:33:01.763 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.926 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:33:01.700 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.932 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:33:01.706 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:48.948 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:48.942+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:48.954 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:48.947+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:49.000 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:33:01.889 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:49.059 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:33:01.944 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:49.096 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:33:01.870 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:45:49.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:49.350+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:49.548 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:49.542+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:49.549 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:49.544+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:49.688 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:49.682+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:49.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:49.734+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:49.759 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:49.754+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:49.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:49.953+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:50.004 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:49.999+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:50.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:50.377+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:50.509 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:50.502+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:50.509 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:50.504+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:50.671 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:50.665+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:50.735 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:50.728+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:50.769 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:50.763+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:50.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:50.950+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:51.022 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:51.016+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:51.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:51.424+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:51.537 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:51.518+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:51.537 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:51.520+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:51.691 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:51.685+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:51.704 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:51.698+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:51.788 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:51.782+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:52.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:51.994+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:52.048 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:52.030+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:52.462 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:52.455+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:52.490 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:52.483+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:52.497 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:52.485+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:52.662 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:52.656+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:52.734 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:52.727+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:52.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:52.782+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:53.035 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:53.029+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:53.039 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:53.033+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:53.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:53.429+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:53.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:53.499+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:53.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:53.502+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:53.690 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:53.682+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:53.744 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:53.738+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:53.798 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:53.791+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:53.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:53.794+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:54.025 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:54.019+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:54.085 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:54.069+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:54.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:54.424+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:54.497 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:54.492+0000 1694c700 -1 osd.3 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.923760+0000 front 2019-11-29T16:45:00.595713+0000 (oldest deadline 2019-11-29T16:45:44.085302+0000) 2019-11-29T16:45:54.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:54.494+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:54.681 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:54.675+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:54.796 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:54.789+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:54.839 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:54.833+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:54.840 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:54.836+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:55.078 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:55.072+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:55.085 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:55.079+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:55.482 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:55.476+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:55.490 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:55.486+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:55.640 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:55.634+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:55.828 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:55.823+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:55.874 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:55.867+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:55.874 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:55.870+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:56.058 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:56.051+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:56.101 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:56.096+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:56.462 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:56.458+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:56.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:56.517+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:56.631 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:56.625+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:56.870 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:56.863+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:56.904 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:56.897+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:56.904 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:56.900+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:57.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:57.012+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:57.084 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:57.078+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:57.434 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:57.431+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:57.490 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:57.484+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:57.613 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:57.607+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:57.869 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:57.863+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:57.915 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:57.910+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:57.917 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:57.913+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:58.043 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:58.037+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:58.057 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:58.051+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:58.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:58.425+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:58.471 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:58.465+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:58.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:58.595+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:58.897 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:58.893+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:58.898 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:58.895+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:58.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:58.915+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:59.031 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:59.026+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:45:59.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:45:59.099+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:45:59.469 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:45:59.458+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:45:59.469 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:45:59.463+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:45:59.609 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:45:59.604+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:45:59.862 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:59.856+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:45:59.865 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:45:59.861+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:45:59.918 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:45:59.912+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:45:59.999 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:45:59.994+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:00.093 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:00.087+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:00.421 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:00.416+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:00.469 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:00.465+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:00.658 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:00.652+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:00.850 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:00.844+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:00.852 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:00.846+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:00.912 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:00.905+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:00.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:00.959+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:01.133 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:01.126+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:01.434 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:01.429+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:01.523 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:01.519+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:01.647 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:01.641+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:01.902 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:01.895+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:01.903 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:01.897+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:01.929 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:01.923+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:02.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:02.008+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:02.174 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:02.168+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:02.437 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:02.428+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:02.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:02.494+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:02.653 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:02.635+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:02.911 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:02.905+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:02.915 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:02.909+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:02.916 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:02.912+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:03.037 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:03.031+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:03.195 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:03.190+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:03.399 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:03.393+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:03.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:03.450+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:03.617 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:03.611+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:03.890 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:03.883+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:03.933 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:03.928+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:03.935 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:03.931+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:04.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:04.011+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:04.229 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:04.223+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:04.436 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:04.431+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:04.439 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:04.436+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:04.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:04.617+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:04.870 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:04.864+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:04.908 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:04.902+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:04.910 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:04.905+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:05.033 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:05.027+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:05.230 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:05.225+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:05.417 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:05.414+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:05.439 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:05.435+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:05.643 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:05.637+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:05.859 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:05.852+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:05.926 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:05.921+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:05.929 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:05.925+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:06.069 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:06.062+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:06.280 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:06.274+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:06.391 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:06.387+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:06.399 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:06.393+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:06.639 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:06.632+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:06.842 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:06.836+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:06.936 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:06.931+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:06.937 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:06.932+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:07.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:07.022+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:07.271 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:07.266+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:07.357 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:07.352+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:07.392 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:07.389+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:07.596 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:07.589+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:07.839 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:07.833+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:07.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:07.948+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:07.954 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:07.951+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:08.033 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:08.024+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:08.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:08.317+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:08.374 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:08.367+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:08.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:08.377+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:08.628 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:08.621+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:08.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:08.842+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:08.988 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:08.981+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:08.988 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:08.984+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:09.020 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:09.014+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:09.315 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:09.306+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:09.358 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:09.354+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:09.370 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:09.364+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:09.596 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:09.590+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:09.862 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:09.855+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:09.992 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:09.983+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:09.992 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:09.988+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:10.062 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:10.056+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:10.365 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:10.359+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:10.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:10.373+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:10.396 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:10.392+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:10.591 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:10.585+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:10.847 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:10.841+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:10.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:10.992+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:11.002 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:10.997+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:11.045 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:11.040+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:11.378 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:11.374+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:11.387 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:11.378+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:11.411 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:11.406+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:11.581 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:11.574+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:11.835 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:11.829+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:11.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:11.991+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:11.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:11.993+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:12.007 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:12.002+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:12.382 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:12.376+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:12.411 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:12.405+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:12.417 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:12.414+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:12.588 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:12.582+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:12.835 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:12.827+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:13.031 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:13.024+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:13.033 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:13.027+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:13.042 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:13.036+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:13.308 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:46:13.305+0000 20e22700 -1 RGWWatcher::handle_error cookie 823873392 err (107) Transport endpoint is not connected 2019-11-29T16:46:13.367 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:13.363+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:13.411 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:13.406+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:13.450 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:13.443+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:13.613 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:13.606+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:13.811 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:13.803+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:13.857 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:46:13.852+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e209 v209) 2019-11-29T16:46:13.990 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:13.986+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:13.992 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:13.989+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:14.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:14.000+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:14.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:14.320+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:14.369 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:14.365+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:14.448 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:14.441+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:14.606 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:14.598+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:14.812 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:14.805+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:14.972 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:14.966+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:15.044 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:15.039+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:15.044 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:15.041+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:15.343 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:15.339+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:15.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:15.419+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:15.454 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:15.448+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:15.631 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:15.623+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:15.824 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:15.817+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:16.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:15.997+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:16.023 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:16.015+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:16.024 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:16.019+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:16.331 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:16.326+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:16.429 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:16.422+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:16.435 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:16.431+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:16.588 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:16.581+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:16.860 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:16.853+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:16.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:16.954+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:17.034 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:17.022+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:17.034 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:17.024+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:17.351 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:17.346+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:17.401 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:17.394+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:17.439 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:17.435+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:17.558 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:17.551+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:17.814 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:17.808+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:17.949 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:17.942+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:18.070 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:18.065+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:18.075 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:18.070+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:18.361 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:18.355+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:18.415 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:18.408+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:18.462 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:18.458+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:18.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:18.594+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:18.835 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:18.828+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:18.878 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:46:18.871+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e209 v209) 2019-11-29T16:46:18.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:18.925+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:19.085 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:19.080+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:19.086 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:19.082+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:19.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:19.328+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:19.372 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:19.366+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:19.500 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:19.496+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:19.604 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:19.598+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:19.841 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:19.833+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:19.958 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:19.951+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:20.054 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:20.049+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:20.055 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:20.050+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:20.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:20.374+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:20.384 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:20.377+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:20.518 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:20.513+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:20.594 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:20.586+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:20.893 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:20.886+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:20.944 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:20.937+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:21.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:21.037+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:21.048 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:21.041+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:21.389 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:21.383+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:21.397 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:21.390+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:21.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:21.486+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:21.575 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:21.568+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:21.895 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:21.888+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:21.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:21.956+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:22.012 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:22.006+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:22.015 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:22.009+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:22.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:22.374+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:22.404 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:22.398+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:22.523 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:22.518+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:22.562 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:22.557+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:22.944 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:22.938+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:22.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:22.941+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:22.977 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:22.970+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:22.985 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:22.975+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:23.360 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:46:23.355+0000 1694c700 -1 osd.6 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.732215+0000 front 2019-11-29T16:44:58.634093+0000 (oldest deadline 2019-11-29T16:45:43.296705+0000) 2019-11-29T16:46:23.377 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:46:23.372+0000 1694c700 -1 osd.2 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.249650+0000 front 2019-11-29T16:45:00.840773+0000 (oldest deadline 2019-11-29T16:45:40.819473+0000) 2019-11-29T16:46:23.520 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:46:23.514+0000 1694c700 -1 osd.7 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.587099+0000 front 2019-11-29T16:45:00.892601+0000 (oldest deadline 2019-11-29T16:45:44.057052+0000) 2019-11-29T16:46:23.548 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:23.543+0000 1694c700 -1 osd.3 209 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:23.909 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:46:23.904+0000 1694c700 -1 osd.5 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:45:00.623655+0000 front 2019-11-29T16:45:00.925236+0000 (oldest deadline 2019-11-29T16:45:42.882156+0000) 2019-11-29T16:46:23.931 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:46:23.923+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e209 v209) 2019-11-29T16:46:23.977 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:46:23.971+0000 1694c700 -1 osd.4 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:44:58.954936+0000 front 2019-11-29T16:44:58.808924+0000 (oldest deadline 2019-11-29T16:45:43.349917+0000) 2019-11-29T16:46:23.981 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:23.972+0000 1694c700 -1 osd.0 209 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:45:13.650951+0000 (oldest deadline 2019-11-29T16:45:53.650951+0000) 2019-11-29T16:46:23.981 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:46:23.976+0000 1694c700 -1 osd.0 209 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:99690 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e208) 2019-11-29T16:46:24.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:24.732+0000 1694c700 -1 osd.3 210 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:25.819 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:46:25.788+0000 1694c700 -1 osd.3 211 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80638 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e207) 2019-11-29T16:46:53.660 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:46:53.364+0000 34a72700 -1 osd.1 215 build_incremental_map_msg missing incremental map 214 2019-11-29T16:47:15.321 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:34:28.208 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.326 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:34:28.215 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.338 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:34:28.226 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.342 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:34:28.231 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.351 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:34:28.125 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.359 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:34:28.133 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.530 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:34:28.418 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.539 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:34:28.428 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:15.562 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:34:28.337 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:47:20.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:47:20.544+0000 1694c700 -1 osd.0 215 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=27 ondisk+retry+read+known_if_redirected e212) 2019-11-29T16:48:13.205 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:13.200+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:13.697 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:13.690+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:13.748 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:13.736+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:14.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:14.244+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:14.343 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:14.335+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:14.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:14.560+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:14.700 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:14.690+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:14.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:14.755+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:14.880 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:14.874+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:15.298 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:15.291+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:15.301 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:15.295+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:15.579 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:15.572+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:15.743 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:15.735+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:15.794 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:15.787+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:15.899 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:15.894+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:16.280 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:16.272+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:16.337 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:16.331+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:16.617 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:16.612+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:16.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:16.699+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:16.832 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:16.825+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:16.920 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:16.915+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:17.294 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:17.288+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:17.353 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:17.347+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:17.661 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:17.653+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:17.670 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:17.664+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:17.796 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:17.790+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:17.963 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:17.957+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:18.187 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:18.180+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:18.325 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:18.319+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:18.338 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:18.331+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:18.679 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:18.672+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:18.683 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:18.677+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:18.798 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:18.792+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:19.002 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:18.997+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:19.171 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:19.161+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:19.321 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:19.315+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:19.325 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:19.319+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:19.668 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:19.661+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:19.678 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:19.671+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:19.794 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:19.788+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:19.956 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:19.951+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:20.194 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:20.188+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:20.341 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:20.334+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:20.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:20.343+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:20.651 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:20.645+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:20.722 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:20.716+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:20.766 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:20.760+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:20.925 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:20.920+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:21.184 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:21.178+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:21.305 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:21.300+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:21.362 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:21.356+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:21.683 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:21.677+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:21.761 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:21.755+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:21.775 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:21.769+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:21.919 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:21.913+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:22.163 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:22.158+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:22.303 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:22.297+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:22.333 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:22.328+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:22.694 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:22.687+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:22.756 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:22.745+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:22.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:22.751+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:22.940 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:22.935+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:23.119 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:23.115+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:23.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:23.344+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:23.366 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:23.360+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:23.697 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:23.692+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:23.745 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:23.739+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:23.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:23.742+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:23.991 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:23.987+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:24.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:24.080+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:24.323 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:24.317+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:24.358 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:24.351+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:24.721 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:24.714+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:24.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:24.741+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:24.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:24.757+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:24.975 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:24.970+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:25.085 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:25.079+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:25.353 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:25.348+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:25.408 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:25.396+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:25.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:25.725+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:25.775 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:25.769+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:25.814 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:25.807+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:26.028 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:26.022+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:26.080 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:26.074+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:26.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:26.355+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:26.416 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:26.402+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:26.719 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:26.714+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:26.774 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:26.767+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:26.814 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:26.808+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:27.058 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:27.053+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:27.133 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:27.127+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:27.357 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:27.352+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:27.414 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:27.407+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:27.770 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:27.763+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:27.795 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:27.788+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:27.841 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:27.835+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:28.043 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:28.038+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:28.160 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:28.155+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:28.314 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:28.308+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:28.469 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:28.462+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:28.755 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:28.749+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:28.796 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:28.790+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:28.887 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:28.881+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:29.084 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:29.079+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:29.119 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:29.113+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:29.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:29.329+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:29.520 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:29.514+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:29.764 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:29.757+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:29.841 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:29.827+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:29.910 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:29.905+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:30.123 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:30.119+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:30.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:30.137+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:30.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:30.355+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:30.563 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:30.557+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:30.769 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:30.763+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:30.871 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:30.865+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:30.967 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:30.957+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:31.118 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:31.113+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:31.140 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:31.134+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:31.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:31.343+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:31.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:31.544+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:31.825 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:31.819+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:31.864 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:31.857+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:31.953 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:31.947+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:32.125 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:32.118+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:32.128 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:32.120+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:32.331 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:32.324+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:32.567 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:32.561+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:32.828 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:32.822+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:32.846 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:32.839+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:32.972 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:32.966+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:33.134 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:33.129+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:33.149 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:33.145+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:33.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:33.279+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:33.553 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:33.548+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:33.831 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:33.823+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:33.833 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:33.827+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:33.931 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:33.925+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:34.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:34.096+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:34.158 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:34.153+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:34.245 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:34.239+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:34.553 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:34.548+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:34.813 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:34.807+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:34.857 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:34.850+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:34.945 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:34.940+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:35.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:35.091+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:35.193 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:35.187+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:35.230 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:35.224+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:35.512 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:35.506+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:35.773 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:35.767+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:35.846 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:35.839+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:35.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:35.927+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:36.079 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:36.075+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:36.164 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:36.158+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:36.223 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:36.218+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:36.515 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:36.508+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:36.811 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:36.804+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:36.884 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:36.878+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:36.921 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:36.915+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:37.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:37.058+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:37.135 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:37.130+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:37.253 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:37.247+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:37.493 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:37.487+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:37.820 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:37.811+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:37.944 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:37.938+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:37.945 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:37.932+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:38.112 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:38.107+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:38.159 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:38.153+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:38.210 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:38.203+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:38.469 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:38.462+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:38.856 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:38.851+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:38.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:38.914+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:38.966 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:38.958+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:39.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:39.093+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:39.175 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:39.170+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:39.186 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:39.180+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:39.519 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:39.513+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:39.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:39.881+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:39.890 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:39.883+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:40.016 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:40.010+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:40.118 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:40.111+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:40.157 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:40.150+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:40.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:40.205+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:40.524 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:40.519+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:40.879 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:40.873+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:40.927 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:40.921+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:40.994 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:40.988+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:41.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:41.086+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:41.141 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:41.135+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:41.243 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:41.237+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:41.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:41.549+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:41.851 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:41.844+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:41.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:41.955+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:42.003 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:41.996+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:42.130 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:42.124+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:42.171 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:42.165+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:42.243 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:42.236+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:42.554 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:42.547+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:42.826 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:42.819+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:42.923 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:42.917+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:43.029 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:43.024+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:43.137 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:43.129+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:43.147 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:43.140+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:43.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:43.275+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:43.561 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:43.556+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:43.796 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:43.790+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:43.953 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:43.935+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:44.063 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:44.056+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:44.166 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:44.159+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:44.174 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:44.168+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:44.276 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:44.270+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:44.554 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:48:44.549+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 46sec e215 v215) 2019-11-29T16:48:44.615 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:44.608+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:44.754 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:44.747+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:44.905 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:44.895+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:45.094 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:45.086+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:45.169 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:45.162+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:45.204 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:45.199+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:45.323 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:45.318+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:45.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:45.619+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:45.782 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:45.775+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:45.912 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:45.905+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:46.129 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:46.122+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:46.131 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:46.124+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:46.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:46.197+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:46.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:46.319+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:46.658 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:46.651+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:46.767 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:46.760+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:46.870 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:46.863+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:47.086 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:47.080+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:47.133 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:47.126+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:47.214 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:47.209+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:47.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:47.288+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:47.614 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:47.607+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:47.781 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:47.774+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:47.847 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:47.841+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:48.058 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:48.049+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:48.104 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:48.096+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:48.226 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:48.220+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:48.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:48.283+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:48.667 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:48.660+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:48:48.809 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:48:48.802+0000 1694c700 -1 osd.4 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.538722+0000 front 2019-11-29T16:47:31.554133+0000 (oldest deadline 2019-11-29T16:48:13.070377+0000) 2019-11-29T16:48:48.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:48:48.818+0000 1694c700 -1 osd.6 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.019643+0000 front 2019-11-29T16:47:31.854925+0000 (oldest deadline 2019-11-29T16:48:14.449953+0000) 2019-11-29T16:48:49.031 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:48:49.024+0000 1694c700 -1 osd.1 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.000363+0000 front 2019-11-29T16:47:31.946373+0000 (oldest deadline 2019-11-29T16:48:13.951711+0000) 2019-11-29T16:48:49.126 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:48:49.119+0000 1694c700 -1 osd.7 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:31.474558+0000 front 2019-11-29T16:47:31.472512+0000 (oldest deadline 2019-11-29T16:48:13.088850+0000) 2019-11-29T16:48:49.215 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:48:49.209+0000 1694c700 -1 osd.3 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:32.072971+0000 front 2019-11-29T16:47:32.075209+0000 (oldest deadline 2019-11-29T16:48:17.949294+0000) 2019-11-29T16:48:49.275 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:48:49.268+0000 1694c700 -1 osd.2 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:30.111359+0000 front 2019-11-29T16:47:30.774663+0000 (oldest deadline 2019-11-29T16:48:13.011645+0000) 2019-11-29T16:48:49.636 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:48:49.627+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 46sec e215 v215) 2019-11-29T16:48:49.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:48:49.646+0000 1694c700 -1 osd.5 215 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:47:28.582194+0000 front 2019-11-29T16:47:28.216145+0000 (oldest deadline 2019-11-29T16:48:14.093078+0000) 2019-11-29T16:49:22.981 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:22.957+0000 1694c700 -1 osd.7 220 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:24.152 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:24.096+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:25.119 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:25.113+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:26.085 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:26.081+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:27.160 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:27.154+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:28.133 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:28.128+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:29.128 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:29.122+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:30.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:30.113+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:31.107 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:31.101+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:32.119 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:32.109+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:33.073 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:33.069+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:34.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:34.091+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:35.150 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:35.145+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:36.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:36.134+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:37.141 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:37.136+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:38.190 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:38.185+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:39.222 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:39.217+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:40.262 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:40.252+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:41.266 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:41.262+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:42.292 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:42.288+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:43.347 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:43.343+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:44.362 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:44.357+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:45.350 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:45.345+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:46.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:46.389+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:47.396 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:47.392+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:48.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:48.375+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:49.416 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:49.411+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:49.706 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:49:49.690+0000 20e22700 -1 RGWWatcher::handle_error cookie 1050128080 err (107) Transport endpoint is not connected 2019-11-29T16:49:50.451 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:50.447+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:51.499 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:51.495+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:52.501 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:52.486+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:53.462 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:53.458+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:54.422 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:54.417+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:54.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:54.627+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:49:55.434 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:55.428+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:55.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:55.665+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:49:56.400 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:56.396+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:56.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:56.651+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:49:57.357 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:57.352+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:57.670 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:57.664+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:49:58.348 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:58.334+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:58.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:58.683+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:49:59.382 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:49:59.378+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:49:59.683 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:49:59.679+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:00.353 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:00.348+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:00.691 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:00.686+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:01.310 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:01.304+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:01.705 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:01.700+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:02.343 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:02.338+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:02.752 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:02.747+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:03.397 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:03.390+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:03.747 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:03.742+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:04.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:04.392+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:04.724 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:04.720+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:05.366 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:05.361+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:05.763 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:05.758+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:06.400 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:06.395+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:06.773 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:06.769+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:07.356 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:07.350+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:07.771 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:07.766+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:08.341 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:08.327+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:08.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:08.742+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:09.361 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:09.356+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:09.794 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:09.785+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:10.336 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:10.331+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:10.800 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:10.792+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:11.355 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:11.350+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:11.813 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:11.808+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:12.346 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:12.342+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:12.794 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:12.789+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:13.344 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:13.338+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:13.778 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:13.773+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:14.366 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:14.362+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:14.832 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:14.827+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:15.321 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:15.317+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:15.807 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:15.801+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:16.040 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:50:16.032+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414644864 err (107) Transport endpoint is not connected 2019-11-29T16:50:16.304 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:16.300+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:16.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:16.780+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:17.348 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:17.343+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:17.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:17.766+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:18.361 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:18.357+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:18.790 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:18.783+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:19.337 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:19.333+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:19.801 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:19.794+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:20.031 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:20.027+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:20.353 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:20.349+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:20.792 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:20.775+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:20.995 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:20.986+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:21.388 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:21.383+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:21.774 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:21.768+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:21.949 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:21.944+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:22.358 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:22.351+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:22.784 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:22.779+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:22.915 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:22.910+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:23.374 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:23.370+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:23.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:23.743+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:23.894 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:23.889+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:24.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:24.388+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:24.805 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:24.795+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:24.890 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:24.885+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:25.367 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:25.361+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:25.815 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:25.809+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:25.889 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:25.882+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:26.343 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:26.339+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:26.833 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:26.823+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:26.873 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:26.868+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:27.302 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:27.297+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:27.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:27.794+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:27.882 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:27.878+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:28.355 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:28.349+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:28.782 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:28.777+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:28.931 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:28.918+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:29.378 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:29.374+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:29.802 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:29.797+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:29.927 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:29.923+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:30.416 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:30.412+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:30.793 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:30.788+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:30.893 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:30.889+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:31.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:31.424+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:31.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:31.793+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:31.801 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:31.796+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:31.878 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:31.873+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:32.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:32.381+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:32.850 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:32.844+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:32.852 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:32.846+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:32.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:32.914+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:33.358 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:33.354+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:33.693 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:33.688+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:33.885 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:33.878+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:34.058 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:33.881+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:34.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:33.895+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:34.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:33.897+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:34.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:34.318+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:34.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:34.320+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:34.585 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:34.579+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:34.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:34.662+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:34.871 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:34.866+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:34.872 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:34.867+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:34.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:34.901+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:34.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:34.904+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:35.347 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:35.340+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:35.347 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:35.343+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:35.607 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:35.602+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:35.648 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:35.640+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:35.856 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:35.849+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:35.856 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:35.852+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:35.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:35.907+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:35.914 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:35.909+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:36.392 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:36.385+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:36.395 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:36.390+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:36.506 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:36.499+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:36.609 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:36.602+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:36.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:36.662+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:36.894 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:36.888+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:36.895 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:36.890+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:36.946 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:36.930+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:36.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:36.944+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:36.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:36.946+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:37.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:37.366+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:37.374 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:37.369+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:37.526 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:37.520+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:37.621 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:37.617+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:37.707 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:37.702+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:37.907 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:37.902+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:37.909 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:37.905+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:37.915 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:37.911+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:37.918 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:37.914+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:37.951 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:37.944+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:38.330 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:38.322+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:38.331 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:38.325+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:38.530 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:38.518+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:38.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:38.583+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:38.682 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:38.675+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:38.902 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:38.896+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:38.903 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:38.898+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:38.941 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:38.933+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:38.942 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:38.937+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:38.988 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:38.980+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:39.327 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:39.320+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:39.328 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:39.323+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:39.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:39.539+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:39.567 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:39.563+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:39.704 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:39.698+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:39.905 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:39.900+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:39.906 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:39.902+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:39.971 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:39.963+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:39.972 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:39.967+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:39.974 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:39.970+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:40.282 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:40.275+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:40.283 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:40.278+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:40.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:40.544+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:40.597 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:40.590+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:40.700 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:40.693+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:40.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:40.859+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:40.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:40.861+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:40.993 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:40.986+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:40.994 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:40.989+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:41.015 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:41.009+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:41.273 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:41.265+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:41.273 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:41.268+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:41.567 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:41.562+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:41.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:41.590+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:41.656 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:41.650+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:41.833 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:41.827+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:41.834 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:41.830+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:41.946 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:41.941+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:41.951 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:41.946+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:42.015 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:42.008+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:42.244 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:42.236+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:42.244 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:42.239+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:42.539 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:42.533+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:42.619 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:42.611+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:42.649 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:42.642+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:42.815 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:42.805+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:42.816 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:42.809+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:42.907 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:42.902+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:42.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:42.904+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:42.981 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:42.968+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:43.287 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:43.277+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:43.287 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:43.280+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:43.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:43.511+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:43.652 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:43.643+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:43.667 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:43.660+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:43.809 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:43.803+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:43.811 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:43.806+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:43.939 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:43.931+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:43.940 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:43.935+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:43.979 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:43.972+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:44.303 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:44.297+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:44.303 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:44.298+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:44.488 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:44.483+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:44.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:44.647+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:44.704 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:44.697+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:44.783 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:44.778+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:44.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:44.782+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:44.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:44.903+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:44.911 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:44.905+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:45.010 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:45.004+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:45.303 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:45.296+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:45.305 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:45.299+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:45.476 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:45.471+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:45.664 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:45.656+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:45.734 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:45.727+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:45.780 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:45.775+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:45.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:45.779+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:45.929 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:45.924+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:45.935 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:45.928+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:46.021 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:46.014+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:46.303 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:46.295+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:46.306 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:46.299+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:46.474 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:46.469+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:46.668 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:46.661+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:46.720 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:46.713+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:46.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:46.768+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:46.776 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:46.771+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:46.894 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:46.887+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:46.895 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:46.890+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:47.019 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:47.012+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:47.347 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:47.340+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:47.350 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:47.345+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:47.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:47.426+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:47.660 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:47.654+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:47.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:47.725+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:47.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:47.728+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:47.733 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:47.723+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:47.876 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:47.871+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:47.880 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:47.875+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:48.052 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:48.044+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:48.315 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:48.308+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:48.316 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:48.311+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:48.393 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:48.386+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:48.678 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:48.672+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:48.700 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:48.694+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:48.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:48.725+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:48.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:48.728+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:48.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:48.858+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:48.868 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:48.863+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:49.050 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:49.044+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:49.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:49.341+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:49.369 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:49.363+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:49.370 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:49.366+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:49.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:49.663+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:49.693 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:49.683+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:49.693 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:49.687+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:49.751 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:49.745+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:49.894 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:49.882+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:49.894 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:49.886+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:50.044 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:50.038+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:50.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:50.327+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:50.423 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:50.418+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:50.426 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:50.422+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:50.630 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:50.618+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:50.652 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:50.646+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:50.655 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:50.650+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:50.795 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:50.789+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:50.890 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:50.883+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:50.892 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:50.886+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:51.052 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:51.046+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:51.373 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:51.368+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:51.387 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:51.382+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:51.389 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:51.386+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:51.609 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:51.597+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:51.657 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:51.649+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:51.661 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:51.653+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:51.839 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:51.832+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:51.852 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:51.847+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:51.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:51.852+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:52.012 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:52.006+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:52.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:52.374+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:52.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:52.376+0000 1694c700 -1 osd.7 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100036 2.33s0 2:ce1f792a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.616_%e9%90%8b%e9%90%8c%e8%94%88%e0%a4%83%e0%a4%80%ef%84%8f%e1%a0%84%e3%9c%84%e1%a0%84%ed%88%8d%e8%94%8b%e3%9c%84%e2%98%83%ce%a9%ef%84%92:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-11-29T16:50:52.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:52.385+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:52.568 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:52.562+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:52.692 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:52.687+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:52.713 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:52.692+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:52.823 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:52.817+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:52.863 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:52.857+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:52.865 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:52.859+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:53.013 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:50:53.007+0000 1694c700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.615500+0000 front 2019-11-29T16:49:51.712701+0000 (oldest deadline 2019-11-29T16:50:36.875525+0000) 2019-11-29T16:50:53.360 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:53.353+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:53.422 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:53.415+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:53.598 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:53.592+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:53.743 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:53.736+0000 1694c700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.511135+0000 front 2019-11-29T16:49:51.480006+0000 (oldest deadline 2019-11-29T16:50:33.349135+0000) 2019-11-29T16:50:53.744 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:53.738+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:53.778 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:53.772+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:53.855 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:53.850+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:53.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:53.851+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:54.342 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:50:54.211+0000 1694c700 -1 osd.1 220 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80502 7.9 7:9195808b:::obj_delete_at_hint.0000000027:head [call lock.lock in=50b] snapc 0=[] RETRY=28 ondisk+retry+write+known_if_redirected e216) 2019-11-29T16:50:54.411 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:54.405+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:54.428 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:54.422+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:54.573 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:54.566+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:54.774 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:50:54.768+0000 1694c700 -1 osd.6 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:48.387683+0000 front 2019-11-29T16:49:48.385080+0000 (oldest deadline 2019-11-29T16:50:33.565906+0000) 2019-11-29T16:50:54.793 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:50:54.787+0000 1694c700 -1 osd.3 221 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100131 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414644864 gen 1] snapc 0=[] ondisk+write+known_if_redirected e221) 2019-11-29T16:50:54.837 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:54.830+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:54.840 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:54.835+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:55.433 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:50:55.419+0000 1694c700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.511525+0000 front 2019-11-29T16:49:50.727700+0000 (oldest deadline 2019-11-29T16:50:33.784637+0000) 2019-11-29T16:50:55.456 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:50:55.452+0000 1694c700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:50.341590+0000 front 2019-11-29T16:49:50.297901+0000 (oldest deadline 2019-11-29T16:50:33.690761+0000) 2019-11-29T16:50:55.597 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:50:55.593+0000 1694c700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.512116+0000 front 2019-11-29T16:49:51.523697+0000 (oldest deadline 2019-11-29T16:50:36.223284+0000) 2019-11-29T16:50:56.051 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:56.019+0000 1694c700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:56.182 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:56.177+0000 1694c700 -1 osd.0 221 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:57.221 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:57.209+0000 1694c700 -1 osd.0 222 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:49:51.729569+0000 front 2019-11-29T16:49:47.381148+0000 (oldest deadline 2019-11-29T16:50:31.600059+0000) 2019-11-29T16:50:57.222 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:57.211+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:58.193 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:58.188+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:50:59.161 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:50:59.155+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:00.132 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:00.127+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:01.086 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:01.081+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:02.071 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:02.066+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:03.122 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:03.116+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:04.179 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:04.173+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:05.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:05.165+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:06.291 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:06.285+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:07.306 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:07.298+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:08.323 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:08.318+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:09.379 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:09.373+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:10.423 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:10.417+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:11.480 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:11.466+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:11.565 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:38:24.454 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.569 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:38:24.458 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.578 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:38:24.466 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.583 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:38:24.471 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.586 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:38:24.361 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.590 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:38:24.365 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.851 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:38:24.740 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.857 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:38:24.632 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:11.866 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:38:24.754 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:51:12.505 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:12.501+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:13.514 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:13.509+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:14.513 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:14.508+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:15.562 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:15.557+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:16.586 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:16.581+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:17.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:17.630+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:18.662 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:18.657+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:19.665 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:19.661+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:20.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:20.652+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:21.671 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:21.667+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:22.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:22.647+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:23.633 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:23.628+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:24.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:24.588+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:51:26.116 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:51:26.063+0000 1694c700 -1 osd.0 222 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e220) 2019-11-29T16:52:21.400 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:21.391+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:21.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:21.916+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:22.432 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:22.426+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:22.502 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:22.497+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:22.936 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:22.931+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:22.971 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:22.964+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:23.473 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:23.467+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:23.505 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:23.499+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:23.942 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:23.934+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:23.992 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:23.985+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:24.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:24.501+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:24.548 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:24.542+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:24.923 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:24.917+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:25.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:25.010+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:25.032 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:25.026+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:25.533 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:25.526+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:25.548 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:25.542+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:25.949 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:25.942+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:26.005 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:25.997+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:26.007 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:26.000+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:26.497 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:26.491+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:26.527 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:26.520+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:26.979 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:26.972+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:27.022 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:27.015+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:27.030 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:27.024+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:27.494 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:27.488+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:27.512 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:27.506+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:28.030 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:28.024+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:28.071 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:28.064+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:28.079 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:28.072+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:28.482 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:28.475+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:28.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:28.479+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:28.999 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:28.992+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:29.041 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:29.035+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:29.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:29.087+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:29.472 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:29.465+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:29.483 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:29.476+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:30.040 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:30.033+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:30.046 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:30.038+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:30.080 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:30.073+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:30.475 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:30.469+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:30.491 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:30.484+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:31.025 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:31.019+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:31.035 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:31.028+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:31.051 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:31.045+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:31.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:31.473+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:31.521 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:31.511+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:31.982 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:31.976+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:32.043 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:32.037+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:32.079 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:32.070+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:32.474 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:32.468+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:32.494 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:32.488+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:33.004 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:32.998+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:33.013 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:33.005+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:33.107 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:33.099+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:33.457 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:33.450+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:33.510 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:33.502+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:33.981 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:33.974+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:34.045 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:34.038+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:34.090 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:34.082+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:34.467 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:34.459+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:34.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:34.465+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:35.008 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:35.001+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:35.036 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:35.029+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:35.106 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:35.099+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:35.495 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:35.488+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:35.516 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:35.510+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:36.017 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:36.009+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:36.039 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:36.030+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:36.130 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:36.123+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:36.514 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:36.507+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:36.551 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:36.543+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:37.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:37.011+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:37.055 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:37.048+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:37.122 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:37.112+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:37.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:37.545+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:37.592 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:37.585+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:38.052 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:38.047+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:38.082 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:38.075+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:38.094 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:38.088+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:38.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:38.600+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:38.639 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:38.631+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:39.051 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:39.043+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:39.053 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:39.045+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:39.092 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:39.084+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:39.578 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:39.570+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:39.690 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:39.678+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:40.062 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:40.054+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:40.106 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:40.096+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:40.115 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:40.105+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:40.594 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:40.587+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:40.726 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:40.719+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:41.037 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:41.030+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:41.066 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:41.059+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:41.119 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:41.112+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:41.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:41.619+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:41.712 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:41.706+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:42.005 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:41.997+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:42.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:42.111+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:42.165 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:42.159+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:42.634 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:42.621+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:42.722 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:42.716+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:43.045 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:43.039+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:43.089 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:43.083+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:43.192 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:43.186+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:43.633 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:43.626+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:43.693 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:43.681+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:44.003 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:43.996+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:44.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:44.063+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:44.173 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:44.167+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:44.591 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:44.584+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:44.741 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:44.733+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:45.011 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:45.005+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:45.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:45.024+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:45.219 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:45.214+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:45.575 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:45.568+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:45.759 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:45.752+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:45.983 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:45.976+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:46.022 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:46.015+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:46.202 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:46.197+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:46.599 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:46.593+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:46.734 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:46.728+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:47.021 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:47.016+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:47.027 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:47.020+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:47.214 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:47.208+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:47.634 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:47.627+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:47.708 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:47.701+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:47.990 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:47.985+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:48.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:48.059+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:48.179 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:48.173+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:48.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:48.595+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:48.715 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:48.709+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:49.040 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:49.035+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:49.084 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:49.078+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:49.186 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:49.180+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:49.619 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:49.608+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:49.762 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:49.757+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:50.009 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:50.003+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:50.131 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:50.125+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:50.139 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:50.135+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:50.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:50.593+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:50.812 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:50.806+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:51.047 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:51.041+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:51.109 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:51.103+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:51.152 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:51.146+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:51.590 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:51.585+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:51.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:51.760+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:52.076 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:52.071+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:52.111 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:52.106+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:52.163 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:52.157+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:52.560 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:52.554+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:52.765 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:52.759+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:53.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:53.025+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:53.109 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:53.104+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:53.124 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:53.118+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:53.542 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:53.533+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:53.725 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:53.719+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:54.005 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:53.999+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:54.118 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:54.112+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:54.132 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:54.127+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:54.153 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:52:54.145+0000 1694c700 -1 osd.7 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:36.449367+0000 front 2019-11-29T16:51:36.205319+0000 (oldest deadline 2019-11-29T16:52:53.265108+0000) 2019-11-29T16:52:54.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:54.545+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:54.730 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:52:54.724+0000 1694c700 -1 osd.6 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.948324+0000 front 2019-11-29T16:51:37.622805+0000 (oldest deadline 2019-11-29T16:52:21.049917+0000) 2019-11-29T16:52:55.008 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:55.002+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:52:55.110 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:52:55.104+0000 1694c700 -1 osd.4 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:35.927324+0000 front 2019-11-29T16:51:35.846013+0000 (oldest deadline 2019-11-29T16:52:21.059668+0000) 2019-11-29T16:52:55.172 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:52:55.166+0000 1694c700 -1 osd.1 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:40.186155+0000 front 2019-11-29T16:51:40.204661+0000 (oldest deadline 2019-11-29T16:52:22.469885+0000) 2019-11-29T16:52:55.172 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:52:55.165+0000 1694c700 -1 osd.7 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:36.449367+0000 front 2019-11-29T16:51:36.205319+0000 (oldest deadline 2019-11-29T16:52:53.265108+0000) 2019-11-29T16:52:55.420 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:52:55.415+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e222 v222) 2019-11-29T16:52:55.507 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:52:55.501+0000 1694c700 -1 osd.5 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:37.578986+0000 front 2019-11-29T16:51:37.660052+0000 (oldest deadline 2019-11-29T16:52:21.879833+0000) 2019-11-29T16:52:55.998 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:52:55.994+0000 1694c700 -1 osd.2 222 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:51:38.919442+0000 front 2019-11-29T16:51:40.064141+0000 (oldest deadline 2019-11-29T16:52:24.764621+0000) 2019-11-29T16:53:53.905 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:53.899+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:53.962 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:53.954+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:54.931 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:54.924+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:54.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:54.978+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:55.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:55.921+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:55.981 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:53:55.973+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:53:55.993 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:55.986+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:56.103 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:53:56.090+0000 20e22700 -1 RGWWatcher::handle_error cookie 1474465792 err (107) Transport endpoint is not connected 2019-11-29T16:53:56.436 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:53:56.428+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:53:56.645 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:53:56.636+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:53:56.890 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:56.884+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:56.942 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:53:56.937+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:53:56.969 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:56.963+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:57.411 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:53:57.404+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:53:57.669 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:53:57.661+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:53:57.862 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:57.856+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:57.970 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:57.965+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:57.981 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:53:57.976+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:53:58.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:53:58.419+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:53:58.697 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:53:58.689+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:53:58.735 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:53:58.728+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:53:58.831 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:58.823+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:58.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:58.922+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:58.958 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:53:58.953+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:53:59.418 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:53:59.412+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:53:59.735 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:53:59.729+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:53:59.739 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:53:59.732+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:53:59.804 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:53:59.798+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:53:59.919 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:53:59.913+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:53:59.990 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:53:59.984+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:00.455 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:00.449+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:00.714 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:00.704+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:00.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:00.757+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:00.853 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:00.846+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:00.901 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:00.895+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:01.034 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:01.027+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:01.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:01.423+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:01.481 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:01.472+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:01.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:01.692+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:01.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:01.751+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:01.866 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:01.856+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:01.891 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:01.884+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:02.042 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:02.035+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:02.409 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:02.404+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:02.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:02.477+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:02.669 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:02.661+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:02.794 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:02.788+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:02.833 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:02.826+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:02.857 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:02.851+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:03.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:03.047+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:03.394 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:03.389+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:03.458 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:03.450+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:03.644 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:03.639+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:03.783 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:03.777+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:03.835 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:03.828+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:03.844 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:03.838+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:04.036 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:04.028+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:04.349 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:04.345+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:04.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:04.460+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:04.605 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:04.598+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:04.823 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:04.818+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:04.848 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:04.841+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:04.877 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:04.870+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:05.047 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:05.040+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:05.374 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:05.367+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:05.427 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:05.421+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:05.605 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:05.599+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:05.830 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:05.825+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:05.851 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:05.846+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:05.861 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:05.857+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:06.015 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:06.009+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:06.334 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:06.328+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:06.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:06.429+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:06.576 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:06.568+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:06.825 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:06.818+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:06.858 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:06.851+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:06.861 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:06.855+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:07.044 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:07.038+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:07.386 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:07.378+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:07.415 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:07.409+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:07.578 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:07.565+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:07.845 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:07.837+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:07.885 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:07.878+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:07.897 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:07.891+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:08.063 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:08.057+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:08.418 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:08.410+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:08.419 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:08.414+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:08.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:08.418+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:08.592 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:08.587+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:08.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:08.858+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:08.895 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:08.889+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:08.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:08.912+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:09.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:09.102+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:09.446 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:09.440+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:09.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:09.449+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:09.457 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:09.452+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:09.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:09.596+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:09.870 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:09.865+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:09.921 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:09.911+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:09.955 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:09.950+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:10.128 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:10.122+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:10.459 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:10.452+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:10.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:10.459+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:10.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:10.462+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:10.646 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:10.638+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:10.837 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:10.831+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:10.928 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:10.922+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:11.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:11.003+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:11.146 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:11.133+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:11.428 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:11.421+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:11.432 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:11.423+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:11.511 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:11.504+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:11.666 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:11.660+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:11.830 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:11.823+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:11.959 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:11.954+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:11.977 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:11.972+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:12.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:12.161+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:12.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:12.417+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:12.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:12.420+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:12.541 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:12.525+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:12.640 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:12.633+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:12.788 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:12.781+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:12.969 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:12.950+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:13.015 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:13.006+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:13.161 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:13.156+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:13.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:13.394+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:13.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:13.399+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:13.515 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:13.509+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:13.684 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:13.677+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:13.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:13.735+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:13.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:13.968+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:14.001 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:13.994+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:14.213 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:14.207+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:14.402 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:14.395+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:14.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:14.400+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:14.567 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:14.560+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:14.659 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:14.654+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:14.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:14.722+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:14.982 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:14.976+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:15.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:14.998+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:15.218 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:15.211+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:15.395 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:15.389+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:15.396 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:15.391+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:15.538 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:15.532+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:15.641 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:15.636+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:15.683 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:15.676+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:16.020 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:16.015+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:16.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:16.032+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:16.249 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:16.242+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:16.364 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:16.358+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:16.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:16.360+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:16.575 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:16.569+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:16.614 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:16.608+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:16.674 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:16.669+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:16.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:16.988+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:17.046 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:17.040+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:17.211 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:17.204+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:17.323 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:17.316+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:17.324 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:17.320+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:17.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:17.609+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:17.625 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:17.618+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:17.701 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:17.695+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:18.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:17.998+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:18.101 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:18.094+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:18.261 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:18.255+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:18.307 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:18.299+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:18.307 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:18.302+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:18.427 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:54:18.419+0000 20e22700 -1 RGWWatcher::handle_error cookie 5475958320 err (107) Transport endpoint is not connected 2019-11-29T16:54:18.610 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:18.604+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:18.620 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:18.615+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:18.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:18.674+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:18.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:18.956+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:19.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:19.061+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:19.264 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:19.258+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:19.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:19.261+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:19.296 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:19.290+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:19.598 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:19.592+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:19.656 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:19.637+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:19.686 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:19.677+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:20.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:20.008+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:20.107 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:20.100+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:20.305 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:20.300+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:20.308 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:20.302+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:20.318 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:20.313+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:20.617 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:20.610+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:20.631 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:20.626+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:20.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:20.727+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:21.045 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:21.039+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:21.084 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:21.077+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:21.337 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:21.331+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:21.338 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:21.333+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:21.376 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:21.370+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:21.600 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:21.593+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:21.607 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:21.600+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:21.697 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:21.691+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:21.701 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:21.697+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:22.049 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:22.043+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:22.115 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:22.109+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:22.351 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:22.345+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:22.351 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:22.348+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:22.392 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:22.386+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:22.605 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:22.598+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:22.623 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:22.617+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:22.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:22.726+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:22.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:22.728+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:23.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:23.002+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:23.092 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:23.086+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:23.306 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:23.300+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:23.308 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:23.304+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:23.381 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:23.370+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:23.381 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:23.373+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:23.581 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:23.577+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:23.656 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:23.649+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:23.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:23.739+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:23.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:23.741+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:23.978 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:23.971+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:24.080 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:24.074+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:24.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:24.284+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:24.291 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:24.286+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:24.362 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:24.355+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:24.363 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:24.358+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:24.628 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:24.622+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:24.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:24.696+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:24.744 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:24.737+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:24.745 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:24.739+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:24.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:24.955+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:25.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:25.049+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:25.302 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:25.295+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:25.303 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:25.298+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:25.349 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:25.340+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:25.349 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:25.342+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:25.664 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:54:25.660+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e227 v227) 2019-11-29T16:54:25.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:25.663+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:25.697 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:25.690+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:25.768 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:25.763+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:25.770 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:25.766+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:26.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:26.007+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:26.056 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:26.049+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:26.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:26.283+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:26.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:26.286+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:26.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:26.320+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:26.342 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:26.337+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:26.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:26.651+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:26.674 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:26.670+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:26.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:26.748+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:26.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:26.751+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:27.030 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:27.024+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:27.043 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:27.037+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:27.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:27.319+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:27.326 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:27.322+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:27.346 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:27.338+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:27.346 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:27.340+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:27.660 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:27.653+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:27.664 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:27.652+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:27.781 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:27.775+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:27.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:27.778+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:28.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:27.996+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:28.018 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:28.011+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:28.351 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:28.345+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:28.352 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:28.347+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:28.378 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:28.372+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:28.381 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:28.376+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:28.635 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:28.630+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:28.644 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:28.637+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:28.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:28.787+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:28.795 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:28.790+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:28.977 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:28.970+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:29.008 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:29.001+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:29.336 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:29.328+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:29.337 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:29.332+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:29.361 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:29.351+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:29.361 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:29.356+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:29.606 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:29.601+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:29.682 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:29.675+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:29.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:29.788+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:29.795 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:29.790+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:30.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:29.999+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:30.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:30.055+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:30.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:30.320+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:30.328 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:30.324+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:30.374 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:30.366+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:30.374 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:30.369+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:30.572 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:30.566+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:30.678 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:54:30.674+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e227 v227) 2019-11-29T16:54:30.718 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:30.710+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:30.763 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:30.758+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:30.764 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:30.759+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:31.046 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:31.038+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:31.078 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:31.066+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:31.337 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:31.330+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:31.338 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:31.334+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:31.410 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:31.404+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:31.412 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:31.407+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:31.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:31.545+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:31.752 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:31.748+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:31.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:31.751+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:31.772 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:31.765+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:32.036 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:32.029+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:32.075 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:32.067+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:32.295 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:32.289+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:32.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:32.294+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:32.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:32.362+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:32.372 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:32.367+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:32.561 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:32.555+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:32.764 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:32.758+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:32.764 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:32.760+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:32.783 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:32.776+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:33.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:33.054+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:33.127 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:33.121+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:33.311 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:33.306+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:33.312 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:33.308+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:33.395 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:33.388+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:33.397 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:33.391+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:33.520 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:33.510+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:33.785 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:33.778+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:33.810 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:33.803+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:33.811 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:33.806+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:34.103 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:34.095+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:34.186 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:34.173+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:34.323 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:34.318+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:34.324 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:34.319+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:34.399 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:34.393+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:34.400 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:34.396+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:34.538 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:34.534+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:34.752 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:34.746+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:34.817 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:34.812+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:34.820 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:34.816+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:35.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:35.132+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:35.199 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:35.193+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:35.298 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:35.291+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:35.300 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:35.294+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:35.381 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:35.376+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:35.385 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:35.380+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:35.506 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:35.501+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:35.695 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:54:35.687+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e227 v227) 2019-11-29T16:54:35.712 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:35.706+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:35.817 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:35.812+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:35.818 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:35.814+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:36.176 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:36.169+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:36.247 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:36.241+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:36.291 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:36.286+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:36.294 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:36.290+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:36.437 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:36.431+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:36.440 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:36.436+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:36.484 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:36.477+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:36.763 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:36.757+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:36.784 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:36.777+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:36.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:36.780+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:37.169 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:37.156+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:37.281 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:37.275+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:37.349 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:37.335+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:37.349 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:37.336+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:37.468 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:37.463+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:37.475 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:37.466+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:37.475 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:37.469+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:37.774 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:37.769+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:37.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:37.771+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:37.805 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:37.797+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:38.153 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:38.143+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:38.308 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:38.302+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:38.353 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:38.347+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:38.354 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:38.349+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:38.446 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:38.440+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:38.447 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:38.443+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:38.514 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:38.508+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:38.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:38.785+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:38.791 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:38.786+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:38.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:38.816+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:39.146 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:39.140+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:39.277 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:39.269+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:39.398 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:39.393+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:39.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:39.396+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:39.438 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:39.430+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:39.438 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:39.432+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:39.477 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:39.472+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:39.804 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:39.796+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:39.806 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:39.800+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:39.808 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:39.802+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:40.141 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:54:40.135+0000 1694c700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.389341+0000 front 2019-11-29T16:53:12.710350+0000 (oldest deadline 2019-11-29T16:53:58.243658+0000) 2019-11-29T16:54:40.254 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:54:40.247+0000 1694c700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.472600+0000 front 2019-11-29T16:53:12.484726+0000 (oldest deadline 2019-11-29T16:53:53.775362+0000) 2019-11-29T16:54:40.406 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:40.400+0000 1694c700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.479328+0000 front 2019-11-29T16:53:10.448329+0000 (oldest deadline 2019-11-29T16:53:55.717178+0000) 2019-11-29T16:54:40.413 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:40.405+0000 1694c700 -1 osd.6 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:40.426 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:40.420+0000 1694c700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:53:27.458437+0000 (oldest deadline 2019-11-29T16:54:07.458437+0000) 2019-11-29T16:54:40.427 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:54:40.423+0000 1694c700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e226) 2019-11-29T16:54:40.524 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:54:40.519+0000 1694c700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.470598+0000 front 2019-11-29T16:53:10.026806+0000 (oldest deadline 2019-11-29T16:53:55.871183+0000) 2019-11-29T16:54:40.755 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:54:40.747+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e227 v227) 2019-11-29T16:54:40.800 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:40.796+0000 1694c700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:12.203325+0000 front 2019-11-29T16:53:12.299756+0000 (oldest deadline 2019-11-29T16:53:53.173491+0000) 2019-11-29T16:54:40.801 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:40.798+0000 1694c700 -1 osd.3 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:40.831 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:54:40.826+0000 1694c700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:53:10.531626+0000 front 2019-11-29T16:53:10.529187+0000 (oldest deadline 2019-11-29T16:53:55.757676+0000) 2019-11-29T16:54:41.440 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:41.436+0000 1694c700 -1 osd.6 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:41.787 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:54:41.782+0000 1694c700 -1 osd.3 228 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:100607 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:54:42.500 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:54:42.484+0000 1694c700 -1 osd.6 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:100612 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=215b] snapc 0=[] ondisk+write+known_if_redirected e227) 2019-11-29T16:55:10.745 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:42:23.633 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:10.750 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:42:23.639 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:10.758 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:42:23.646 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:10.768 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:42:23.650 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:10.769 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:42:23.535 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:10.769 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:42:23.541 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:11.002 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:42:23.890 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:11.003 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:42:23.777 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:55:11.031 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:42:23.918 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:28.137 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:28.131+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:28.736 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:28.730+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:29.109 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:29.102+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:29.318 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:29.310+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:29.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:29.380+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:29.790 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:29.784+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:30.127 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:30.120+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:30.127 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:30.120+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:30.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:30.324+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:30.372 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:30.366+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:30.749 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:30.743+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:31.151 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:31.144+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:31.161 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:31.154+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:31.324 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:31.317+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:31.352 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:31.342+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:31.543 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:31.537+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:31.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:31.793+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:32.185 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:32.179+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:32.202 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:32.195+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:32.320 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:32.314+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:32.335 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:32.329+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:32.592 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:32.587+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:32.858 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:32.852+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:32.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:32.974+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:33.181 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:33.175+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:33.229 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:33.223+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:33.284 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:33.277+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:33.373 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:33.367+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:33.557 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:33.552+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:33.878 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:33.872+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:34.022 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:34.015+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:34.169 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:34.164+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:34.269 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:34.264+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:34.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:34.318+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:34.425 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:34.420+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:34.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:34.520+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:34.930 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:34.917+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:35.047 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:35.040+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:35.143 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:35.137+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:35.321 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:35.312+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:35.321 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:35.315+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:35.455 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:35.450+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:35.578 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:35.573+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:35.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:35.897+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:36.012 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:36.005+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:36.172 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:36.166+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:36.283 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:36.278+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:36.303 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:36.298+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:36.462 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:36.456+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:36.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:36.625+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:36.876 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:36.870+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:37.052 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:37.046+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:37.198 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:37.192+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:37.260 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:37.254+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:37.299 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:37.293+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:37.469 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:37.465+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:37.593 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:37.585+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:37.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:37.916+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:38.022 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:38.016+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:38.157 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:38.150+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:38.299 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:38.293+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:38.333 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:38.327+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:38.496 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:38.491+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:38.584 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:38.577+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:38.971 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:38.964+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:39.070 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:39.063+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:39.185 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:39.179+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:39.303 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:39.297+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:39.353 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:39.348+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:39.487 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:39.482+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:39.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:39.554+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:40.004 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:39.994+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:40.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:40.030+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:40.211 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:40.195+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:40.314 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:40.307+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:40.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:40.391+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:40.459 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:40.453+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:40.531 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:40.526+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:40.987 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:40.980+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:41.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:41.038+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:41.224 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:41.218+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:41.280 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:41.270+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:41.374 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:41.368+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:41.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:41.430+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:41.534 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:41.530+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:42.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:42.002+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:42.011 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:42.004+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:42.230 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:42.225+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:42.260 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:42.253+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:42.417 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:42.410+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:42.443 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:42.438+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:42.564 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:42.559+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:42.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:42.990+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:42.998 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:42.993+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:43.211 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:43.206+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:43.266 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:43.260+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:43.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:43.426+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:43.459 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:43.453+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:43.603 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:43.598+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:43.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:43.978+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:43.989 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:43.983+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:44.179 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:44.174+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:44.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:44.268+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:44.438 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:44.432+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:44.464 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:44.457+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:44.603 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:44.598+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:44.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:44.955+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:44.982 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:44.976+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:45.215 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:45.209+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:45.243 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:45.237+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:45.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:45.422+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:45.451 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:45.447+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:45.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:45.616+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:45.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:45.974+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:46.003 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:45.996+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:46.269 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:46.263+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:46.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:46.267+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:46.408 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:46.401+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:46.459 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:46.452+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:46.614 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:46.609+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:46.979 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:46.973+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:47.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:46.996+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:47.259 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:47.252+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:47.289 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:47.284+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:47.389 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:47.382+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:47.424 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:47.420+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:47.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:47.632+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:47.965 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:47.957+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:48.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:48.010+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:48.260 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:48.255+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:48.319 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:48.303+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:48.403 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:48.398+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:48.405 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:48.399+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:48.644 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:48.638+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:48.988 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:48.982+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:48.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:48.993+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:49.225 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:49.219+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:49.322 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:49.316+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:49.382 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:49.374+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:49.442 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:49.436+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:49.635 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:49.630+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:49.950 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:49.944+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:49.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:49.993+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:50.235 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:50.228+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:50.338 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:50.332+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:50.361 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:50.355+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:50.490 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:50.481+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:50.671 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:50.666+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:50.993 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:50.987+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:51.007 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:51.000+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:51.239 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:51.232+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:51.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:51.335+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:51.377 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:51.372+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:51.464 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:51.458+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:51.699 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:51.695+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:51.982 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:51.976+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:52.035 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:52.029+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:52.252 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:52.247+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:52.303 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:52.298+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:52.371 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:52.363+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:52.433 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:52.427+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:52.658 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:52.652+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:52.938 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:52.933+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:53.000 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:52.995+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:53.295 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:53.289+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:53.324 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:53.318+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:53.336 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:53.331+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:53.484 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:53.479+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:53.705 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:53.700+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:53.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:53.895+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:53.968 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:53.963+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:54.291 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:54.285+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:54.336 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:54.329+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:54.342 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:54.337+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:54.513 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:54.506+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:54.730 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:54.725+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:54.882 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:54.876+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:54.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:54.967+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:55.311 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:55.304+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:55.342 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:55.337+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:55.376 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:55.369+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:55.468 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:55.462+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:55.722 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:55.716+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:55.845 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:55.838+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:55.990 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:55.979+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:56.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:56.311+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:56.331 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:56.325+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:56.332 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:56.326+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:56.513 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:56.505+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:56.665 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:44:09.553 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.671 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:44:09.560 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.676 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:44:09.564 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.709 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:44:09.597 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.719 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:56.715+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:56.744 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:44:09.518 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.748 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:44:09.522 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.797 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:44:09.685 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.806 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:44:09.694 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:56.810+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:56.884 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:44:09.658 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:56:56.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:56.948+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:57.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:57.326+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:57.332 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:57.326+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:57.382 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:57.377+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:57.553 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:57.545+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:57.706 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:57.693+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:57.787 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:57.780+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:57.995 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:57.988+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:58.316 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:58.309+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:58.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:58.369+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:58.417 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:58.409+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:58.564 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:58.555+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:58.656 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:58.650+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:58.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:58.816+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:56:59.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:56:59.005+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:56:59.309 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:56:59.301+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:56:59.388 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:56:59.383+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:56:59.389 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:56:59.383+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:56:59.559 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:56:59.552+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:56:59.706 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:56:59.699+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:56:59.846 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:56:59.839+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:00.065 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:00.057+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:00.282 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:00.275+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:00.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:00.370+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:00.425 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:00.419+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:00.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:00.512+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:00.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:00.748+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:00.810 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:00.803+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:01.056 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:01.049+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:01.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:01.316+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:01.404 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:01.397+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:01.473 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:01.467+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:01.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:01.542+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:01.625 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:57:01.621+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e232 v232) 2019-11-29T16:57:01.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:01.783+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:01.841 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:01.834+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:02.071 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:02.063+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:02.336 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:02.328+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:02.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:02.352+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:02.486 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:02.480+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:02.527 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:02.520+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:02.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:02.771+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:02.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:02.805+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:03.052 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:03.045+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:03.315 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:03.309+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:03.356 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:03.349+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:03.534 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:03.527+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:03.589 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:03.580+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:03.814 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:03.808+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:03.838 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:03.831+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:04.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:04.006+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:04.321 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:04.314+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:04.351 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:04.343+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:04.532 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:04.525+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:04.592 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:04.584+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:04.861 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:04.852+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:04.867 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:04.860+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:04.970 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:04.962+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:05.307 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:05.299+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:05.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:05.362+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:05.497 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:05.492+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:05.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:05.594+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:05.868 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:57:05.862+0000 1694c700 -1 osd.4 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.426933+0000 front 2019-11-29T16:55:46.965061+0000 (oldest deadline 2019-11-29T16:56:28.473266+0000) 2019-11-29T16:57:05.918 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:05.909+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:57:05.926 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:57:05.920+0000 1694c700 -1 osd.5 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.860916+0000 front 2019-11-29T16:55:47.744268+0000 (oldest deadline 2019-11-29T16:56:32.242923+0000) 2019-11-29T16:57:06.270 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:57:06.264+0000 1694c700 -1 osd.2 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.440240+0000 front 2019-11-29T16:55:47.439117+0000 (oldest deadline 2019-11-29T16:56:28.500087+0000) 2019-11-29T16:57:06.422 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:57:06.416+0000 1694c700 -1 osd.6 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.912759+0000 front 2019-11-29T16:55:47.242406+0000 (oldest deadline 2019-11-29T16:56:29.398431+0000) 2019-11-29T16:57:06.468 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T16:57:06.461+0000 1694c700 -1 osd.1 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:47.956278+0000 front 2019-11-29T16:55:45.874994+0000 (oldest deadline 2019-11-29T16:56:27.847520+0000) 2019-11-29T16:57:06.611 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:57:06.605+0000 1694c700 -1 osd.7 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:44.006310+0000 front 2019-11-29T16:55:44.051319+0000 (oldest deadline 2019-11-29T16:56:29.287928+0000) 2019-11-29T16:57:06.697 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:57:06.686+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e232 v232) 2019-11-29T16:57:06.884 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:57:06.879+0000 1694c700 -1 osd.3 232 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:55:46.510751+0000 front 2019-11-29T16:55:46.507177+0000 (oldest deadline 2019-11-29T16:56:31.065627+0000) 2019-11-29T16:58:04.389 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:04.383+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:04.737 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:04.730+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:05.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:05.376+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:05.486 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:05.479+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:05.716 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:05.712+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:05.742 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:05.737+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:06.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:06.427+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:06.464 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:06.459+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:06.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:06.458+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:06.711 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:06.706+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:06.727 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:06.721+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:07.394 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:07.388+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:07.469 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:07.463+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:07.489 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:07.482+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:07.501 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:07.496+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:07.748 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:07.741+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:07.779 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:07.773+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:08.410 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:08.403+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:08.425 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:08.419+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:08.501 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:08.495+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:08.556 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:08.550+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:08.762 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:08.757+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:08.831 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:08.825+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:08.897 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:08.891+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:09.408 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:09.401+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:09.450 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:09.444+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:09.528 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:09.518+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:09.552 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:09.546+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:09.793 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:09.786+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:09.814 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:09.810+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:09.950 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:09.944+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:10.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:10.378+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:10.428 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:10.422+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:10.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:10.500+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:10.532 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:10.526+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:10.826 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:10.821+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:10.846 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:10.838+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:10.991 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:10.984+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:11.375 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:11.367+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:11.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:11.425+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:11.527 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:11.518+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:11.538 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:11.532+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:11.836 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:11.831+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:11.863 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:11.857+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:11.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:11.859+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:11.982 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:11.976+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:12.356 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:12.348+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:12.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:12.431+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:12.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:12.488+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:12.517 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:12.511+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:12.799 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:12.793+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:12.892 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:12.885+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:12.894 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:12.889+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:12.961 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:12.954+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:13.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:13.386+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:13.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:13.430+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:13.486 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:13.481+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:13.567 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:13.560+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:13.803 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:13.793+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:13.859 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:13.852+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:13.860 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:13.855+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:13.921 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:13.914+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:14.420 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:14.414+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:14.430 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:14.423+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:14.531 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:14.526+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:14.560 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:14.553+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:14.786 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:14.770+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:14.854 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:14.847+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:14.859 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:14.850+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:14.880 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:14.874+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:15.397 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:15.383+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:15.441 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:15.436+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:15.548 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:15.542+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:15.570 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:15.566+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:15.805 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:15.798+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:15.872 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:15.867+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:15.875 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:15.871+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:15.878 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:15.870+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:16.415 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:16.410+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:16.434 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:16.427+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:16.565 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:16.555+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:16.609 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:16.602+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:16.774 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:16.767+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:16.861 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:16.856+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:16.867 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:16.859+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:16.872 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:16.865+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:17.456 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:17.449+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:17.466 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:17.460+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:17.545 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:17.538+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:17.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:17.615+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:17.762 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:17.752+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:17.825 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:17.814+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:17.825 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:17.817+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:17.908 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:17.901+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:18.432 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:18.425+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:18.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:18.430+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:18.558 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:18.548+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:18.664 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:18.655+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:18.784 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:18.773+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:18.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:18.809+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:18.818 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:18.813+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:18.885 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:18.875+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:19.416 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:19.410+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:19.434 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:19.427+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:19.520 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:19.513+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:19.690 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:19.684+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:19.737 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:19.729+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:19.845 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:19.837+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:19.846 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:19.841+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:19.913 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:19.905+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:20.414 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:20.408+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:20.476 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:20.469+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:20.514 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:20.506+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:20.689 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:20.683+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:20.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:20.715+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:20.891 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:20.883+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:20.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:20.887+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:20.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:20.891+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:21.381 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:21.362+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:21.436 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:21.428+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:21.539 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:21.526+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:21.704 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:21.689+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:21.769 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:21.762+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:21.876 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:21.871+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:21.933 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:21.925+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:21.933 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:21.929+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:22.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:22.386+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:22.406 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:22.401+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:22.518 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:22.512+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:22.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:22.697+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:22.725 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:22.719+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:22.882 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:22.876+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:22.936 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:22.925+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:22.938 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:22.928+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:23.385 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:23.379+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:23.448 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:23.444+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:23.497 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:23.490+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:23.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:23.658+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:23.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:23.659+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:23.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:23.724+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:23.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:23.843+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:23.940 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:23.934+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:23.945 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:23.939+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:24.355 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:24.349+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:24.498 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:24.492+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:24.498 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:24.493+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:24.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:24.655+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:24.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:24.658+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:24.757 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:24.751+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:24.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:24.832+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:24.969 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:24.961+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:24.969 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:24.963+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:25.332 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:25.326+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:25.481 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:25.475+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:25.514 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:25.508+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:25.659 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:25.654+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:25.660 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:25.655+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:25.790 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:25.783+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:25.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:25.832+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:25.966 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:25.960+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:25.968 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:25.963+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:26.288 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:26.282+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:26.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:26.486+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:26.564 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:26.558+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:26.639 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:26.634+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:26.640 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:26.636+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:26.786 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:26.780+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:26.833 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:26.827+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:27.013 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:27.008+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:27.014 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:27.010+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:27.314 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:27.308+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:27.523 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:27.516+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:27.572 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:27.561+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:27.671 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:27.666+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:27.674 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:27.670+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:27.821 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:27.815+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:27.852 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:27.846+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:27.970 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:27.965+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:27.974 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:27.970+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:28.370 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:28.356+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:28.487 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:28.481+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:28.545 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:28.541+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:28.682 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:28.675+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:28.682 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:28.677+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:28.843 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:28.837+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:28.861 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:28.855+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:28.944 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:28.938+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:28.949 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:28.944+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:29.405 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:29.399+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:29.446 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:29.440+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:29.507 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:29.501+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:29.698 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:29.692+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:29.698 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:29.693+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:29.802 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:29.796+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:29.833 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:29.827+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:29.950 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:29.943+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:29.952 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:29.946+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:30.444 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:30.438+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:30.494 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:30.488+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:30.502 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:30.495+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:30.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:30.658+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:30.667 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:30.660+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:30.829 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:30.823+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:30.872 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:30.866+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:30.969 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:30.963+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:30.971 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:30.967+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:31.410 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:31.398+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:31.474 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:31.463+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:31.489 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:31.483+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:31.672 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:31.666+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:31.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:31.668+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:31.854 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:31.849+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:31.869 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:31.863+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:31.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:31.948+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:31.959 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:31.954+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:32.425 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:32.418+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:32.481 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:32.475+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:32.533 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:32.526+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:32.635 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:32.628+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:32.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:32.630+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:32.837 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:32.820+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:32.847 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:32.841+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:32.925 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:32.917+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:32.926 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:32.920+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:33.383 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:33.377+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:33.439 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:33.433+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:33.577 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:33.570+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:33.616 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:33.609+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:33.617 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:33.611+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:33.824 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:33.819+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:33.851 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:33.846+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:33.973 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:33.967+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:33.973 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:33.968+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:34.365 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:34.356+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:34.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:34.418+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:34.582 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:34.576+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:34.605 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:34.598+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:34.606 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:34.600+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:34.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:34.819+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:34.905 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:34.898+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:34.989 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:34.980+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:34.991 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:34.984+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:35.358 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:35.351+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:35.421 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:35.415+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:35.568 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:35.560+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:35.648 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:35.642+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:35.649 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:35.643+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:35.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:35.801+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:35.920 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:35.913+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:35.970 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:35.957+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:35.970 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:35.960+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:36.388 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:36.382+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:36.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:36.417+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:36.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:36.537+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:36.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:36.665+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:36.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:36.668+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:36.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:36.793+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:36.935 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:36.927+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:36.935 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:36.930+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:36.973 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:36.967+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:37.409 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:37.399+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:37.444 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:37.437+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:37.509 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:37.502+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:37.654 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:37.648+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:37.655 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:37.650+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:37.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:37.803+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:37.941 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:37.933+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:37.943 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:37.936+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:37.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:37.988+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:38.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:38.366+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:38.415 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:38.410+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:38.474 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:38.467+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:38.622 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:38.615+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:38.622 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:38.617+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:38.798 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:38.792+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:38.958 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:38.950+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:38.960 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:38.955+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:38.988 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:38.982+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:39.404 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:39.396+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:39.437 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:39.431+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:39.460 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:39.451+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:39.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:39.595+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:39.603 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:39.599+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:39.802 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:39.795+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:39.938 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:39.930+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:39.939 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:39.933+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:39.943 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:39.938+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:40.406 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:40.398+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:40.427 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:40.416+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:40.472 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:40.466+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:40.611 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:40.602+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:40.611 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:40.605+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:40.852 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:40.845+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:40.918 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:40.911+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:40.919 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:40.914+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:40.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:40.974+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:41.392 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T16:58:41.386+0000 1694c700 -1 osd.7 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.481995+0000 front 2019-11-29T16:57:22.499162+0000 (oldest deadline 2019-11-29T16:58:05.933578+0000) 2019-11-29T16:58:41.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T16:58:41.421+0000 1694c700 -1 osd.2 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.893932+0000 front 2019-11-29T16:57:18.935705+0000 (oldest deadline 2019-11-29T16:58:04.142104+0000) 2019-11-29T16:58:41.444 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:41.439+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:41.647 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:41.641+0000 1694c700 -1 osd.3 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.065072+0000 front 2019-11-29T16:57:22.205792+0000 (oldest deadline 2019-11-29T16:58:06.575132+0000) 2019-11-29T16:58:41.647 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:41.642+0000 1694c700 -1 osd.3 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101004 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414673360] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:41.863 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T16:58:41.857+0000 1694c700 -1 osd.6 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.990614+0000 front 2019-11-29T16:57:22.879539+0000 (oldest deadline 2019-11-29T16:58:08.132315+0000) 2019-11-29T16:58:41.942 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:41.937+0000 1694c700 -1 osd.0 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T16:57:31.008929+0000 (oldest deadline 2019-11-29T16:58:11.008929+0000) 2019-11-29T16:58:41.947 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:58:41.941+0000 1694c700 -1 osd.0 238 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-11-29T16:58:41.981 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T16:58:41.970+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 49sec e238 v238) 2019-11-29T16:58:42.016 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T16:58:42.010+0000 1694c700 -1 osd.5 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:18.483160+0000 front 2019-11-29T16:57:18.629262+0000 (oldest deadline 2019-11-29T16:58:03.741188+0000) 2019-11-29T16:58:42.472 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T16:58:42.468+0000 1694c700 -1 osd.4 238 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T16:57:22.501443+0000 front 2019-11-29T16:57:22.500173+0000 (oldest deadline 2019-11-29T16:58:04.765574+0000) 2019-11-29T16:58:42.758 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:42.749+0000 1694c700 -1 osd.3 239 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101025 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 5414673360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:43.951 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T16:58:43.915+0000 1694c700 -1 osd.3 240 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101025 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 5414673360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-11-29T16:58:44.257 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T16:58:44.229+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414673360 err (107) Transport endpoint is not connected 2019-11-29T16:58:47.303 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:00.192 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.310 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:00.199 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.336 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:00.223 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.379 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:00.267 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.413 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:00.187 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.415 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:00.190 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.531 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:00.419 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.570 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:00.344 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:58:47.604 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:00.486 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.518 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:26.407 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.525 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:26.414 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.537 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:26.425 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.542 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:26.430 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.553 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:26.328 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.564 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:26.339 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.589 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:46:26.478 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.601 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:46:26.489 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:13.647 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:46:26.422 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T16:59:48.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T16:59:47.739+0000 1694c700 -1 osd.0 244 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:100704 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~503 in=503b] snapc 0=[] RETRY=4 ondisk+retry+write+known_if_redirected e242) 2019-11-29T17:00:39.136 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:39.129+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:39.518 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:39.511+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:39.695 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:39.690+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:39.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:39.792+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:40.144 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:40.138+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:40.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:40.489+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:40.664 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:40.659+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:40.834 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:40.828+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:41.121 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:41.115+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:41.489 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:41.483+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:41.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:41.620+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:41.858 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:41.853+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:41.860 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:41.842+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:42.082 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:42.075+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:42.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:42.141+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:42.456 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:42.448+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:42.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:42.674+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:42.822 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:42.817+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:42.899 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:42.892+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:43.047 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:43.041+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:43.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:43.141+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:43.446 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:43.438+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:43.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:43.674+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:43.851 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:43.846+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:43.870 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:43.861+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:44.013 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:44.005+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:44.083 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:44.076+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:44.123 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:44.116+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:44.441 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:44.432+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:44.710 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:44.704+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:44.871 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:44.865+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:44.873 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:44.866+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:45.011 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:45.004+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:45.122 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:45.116+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:45.165 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:45.156+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:45.469 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:45.462+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:45.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:45.740+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:45.854 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:45.847+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:45.917 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:45.908+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:45.986 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:00:45.978+0000 1694c700 -1 osd.4 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:58.303977+0000 front 2019-11-29T16:59:58.414691+0000 (oldest deadline 2019-11-29T17:00:38.742883+0000) 2019-11-29T17:00:46.136 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:46.129+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:46.162 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:46.156+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:46.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:46.487+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:46.711 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:46.705+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:46.887 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:46.881+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:46.918 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:46.910+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:47.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:47.133+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:47.204 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:47.199+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:47.542 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:47.536+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:47.675 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:47.670+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:47.850 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:47.843+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:47.925 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:47.918+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:48.180 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:48.172+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:48.251 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:48.244+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:48.525 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:48.518+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:48.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:48.727+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:48.889 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:48.881+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:48.969 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:48.961+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:49.229 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:49.222+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:49.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:49.274+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:49.507 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:49.499+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:49.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:49.768+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:49.849 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:49.843+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:50.001 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:49.993+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:50.284 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:50.276+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:50.302 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:50.296+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:50.466 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:50.460+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:50.749 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:50.742+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:50.895 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:50.888+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:51.028 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:51.021+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:51.314 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:51.306+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:51.322 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:51.315+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:51.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:51.497+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:51.729 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:51.724+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:51.869 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:51.863+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:51.992 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:51.985+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:52.309 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:52.302+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:52.316 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:52.309+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:52.460 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:52.453+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:52.768 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:52.762+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:52.920 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:52.913+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:53.024 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:53.018+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:53.343 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:53.337+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:53.356 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:53.350+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:53.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:53.482+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:53.820 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:53.813+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:53.886 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:53.880+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:54.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:54.064+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:54.311 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:54.304+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:54.412 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:54.406+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:54.542 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:54.536+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:54.840 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:54.834+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:54.889 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:54.884+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:55.065 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:55.059+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:55.329 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:55.323+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:55.431 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:55.426+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:55.555 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:00:55.547+0000 1694c700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.451378+0000 front 2019-11-29T16:59:54.088777+0000 (oldest deadline 2019-11-29T17:00:39.169907+0000) 2019-11-29T17:00:55.877 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:55.870+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:55.924 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:55.915+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:56.079 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:56.073+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:56.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:56.310+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:56.460 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:56.454+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:56.841 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:56.835+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:56.950 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:56.944+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:57.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:57.081+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:57.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:57.335+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:57.430 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:57.424+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:57.890 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:57.885+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:57.970 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:57.963+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:58.072 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:58.066+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:58.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:58.327+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:58.445 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:58.438+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:58.851 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:58.845+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:58.963 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:58.957+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:00:59.135 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:00:59.124+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:00:59.309 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:00:59.302+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:00:59.485 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:00:59.478+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:00:59.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:00:59.832+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:00:59.950 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:00:59.942+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:00.137 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:00.131+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:00.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:00.325+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:00.441 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:00.436+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:00.816 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:00.810+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:00.967 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:00.960+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:01.116 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:01.109+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:01.343 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:01.335+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:01.409 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:01.403+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:01.871 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:01.864+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:01.940 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:01.934+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:02.165 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:02.158+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:02.310 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:02.304+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:02.412 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:02.406+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:02.844 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:02.838+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:02.906 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:02.900+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:03.202 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:03.196+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:03.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:03.286+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:03.404 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:03.397+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:03.804 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:03.798+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:03.910 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:03.904+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:04.177 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:04.170+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:04.257 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:04.250+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:04.385 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:04.380+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:04.799 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:04.792+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:04.919 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:04.912+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:05.220 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:05.214+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:05.224 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:05.219+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:05.375 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:05.368+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:05.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:05.750+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:05.901 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:05.896+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:06.204 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:06.199+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:06.243 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:06.237+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:06.397 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:06.390+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:06.710 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:06.704+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:06.922 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:06.916+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:07.176 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:07.170+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:07.229 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:07.224+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:07.400 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:07.394+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:07.672 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:07.666+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:07.970 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:07.963+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:08.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:08.186+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:08.272 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:08.266+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:08.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:08.446+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:08.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:08.712+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:08.964 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:08.958+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:09.226 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:09.219+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:09.296 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:09.290+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:09.447 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:09.438+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:09.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:09.719+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:09.999 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:09.994+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:10.206 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:10.199+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:10.343 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:10.338+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:10.485 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:10.479+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:10.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:10.731+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:11.033 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:11.026+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:11.257 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:11.251+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:11.301 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:11.296+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:11.486 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:11.481+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:11.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:11.731+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:01:12.077 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:01:12.073+0000 1694c700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.390494+0000 front 2019-11-29T16:59:57.513860+0000 (oldest deadline 2019-11-29T17:00:41.753988+0000) 2019-11-29T17:01:12.263 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:01:12.257+0000 1694c700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.456141+0000 front 2019-11-29T16:59:57.552660+0000 (oldest deadline 2019-11-29T17:00:39.494322+0000) 2019-11-29T17:01:12.315 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:01:12.310+0000 1694c700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.931175+0000 front 2019-11-29T16:59:57.816947+0000 (oldest deadline 2019-11-29T17:00:43.529311+0000) 2019-11-29T17:01:12.464 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:01:12.458+0000 1694c700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:57.404729+0000 front 2019-11-29T16:59:57.479557+0000 (oldest deadline 2019-11-29T17:00:41.631141+0000) 2019-11-29T17:01:12.717 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:01:12.710+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e244 v244) 2019-11-29T17:01:12.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:01:12.781+0000 1694c700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T16:59:54.866708+0000 front 2019-11-29T16:59:54.826497+0000 (oldest deadline 2019-11-29T17:00:38.855734+0000) 2019-11-29T17:02:11.224 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:11.216+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:12.268 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:12.262+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:12.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:12.933+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:13.231 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:13.224+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:13.724 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:13.718+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:13.729 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:13.723+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:13.917 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:13.910+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:14.222 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:14.215+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:14.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:14.741+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:14.749 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:14.743+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:14.955 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:14.948+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:15.160 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:02:15.132+0000 20e22700 -1 RGWWatcher::handle_error cookie 362578592 err (107) Transport endpoint is not connected 2019-11-29T17:02:15.186 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:15.180+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:15.721 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:15.709+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:15.800 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:15.793+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:15.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:15.968+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:16.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:16.192+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:16.674 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:16.667+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:16.820 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:16.812+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:16.987 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:16.981+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:17.232 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:17.226+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:17.671 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:17.665+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:17.810 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:17.803+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:18.003 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:17.997+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:18.197 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:18.191+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:18.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:18.420+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:18.721 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:18.715+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:18.794 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:18.787+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:19.029 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:19.023+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:19.165 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:19.159+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:19.427 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:19.421+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:19.740 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:19.733+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:19.802 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:19.796+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:19.986 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:19.980+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:20.219 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:20.213+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:20.394 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:20.389+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:20.753 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:20.748+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:20.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:20.759+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:20.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:20.950+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:21.180 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:21.175+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:21.433 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:21.428+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:21.738 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:21.731+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:21.795 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:21.789+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:21.943 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:21.938+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:22.215 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:22.208+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:22.467 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:22.462+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:22.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:22.757+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:22.776 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:22.770+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:22.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:22.968+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:23.182 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:23.176+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:23.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:23.418+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:23.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:23.741+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:23.820 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:23.813+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:23.938 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:23.930+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:24.186 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:24.179+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:24.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:24.393+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:24.403 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:24.396+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:24.765 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:24.759+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:24.782 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:24.776+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:24.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:24.866+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:24.945 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:24.930+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:25.170 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:25.165+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:25.421 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:25.414+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:25.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:25.419+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:25.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:25.766+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:25.785 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:25.779+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:25.920 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:25.913+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:25.940 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:25.917+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:26.170 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:26.163+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:26.387 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:26.380+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:26.388 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:26.383+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:26.731 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:26.725+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:26.745 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:26.739+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:26.820 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:02:26.815+0000 20e22700 -1 RGWWatcher::handle_error cookie 1472469456 err (107) Transport endpoint is not connected 2019-11-29T17:02:26.901 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:26.894+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:26.957 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:26.951+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:27.214 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:27.207+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:27.430 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:27.424+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:27.432 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:27.427+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:27.732 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:27.723+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:27.757 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:27.751+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:27.897 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:27.890+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:27.969 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:27.964+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:28.258 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:28.251+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:28.403 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:28.394+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:28.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:28.398+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:28.750 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:28.745+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:28.776 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:28.771+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:28.861 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:28.854+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:29.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:29.009+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:29.243 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:29.237+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:29.416 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:29.410+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:29.421 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:29.415+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:29.746 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:29.738+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:29.770 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:29.764+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:29.884 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:29.879+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:30.060 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:30.053+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:30.229 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:30.223+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:30.452 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:30.443+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:30.453 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:30.447+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:30.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:30.704+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:30.730 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:30.724+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:30.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:30.880+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:31.054 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:31.047+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:31.208 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:31.201+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:31.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:31.326+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:31.436 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:31.430+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:31.439 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:31.434+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:31.755 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:31.749+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:31.770 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:31.763+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:31.877 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:31.859+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:32.060 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:32.053+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:32.171 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:32.165+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:32.317 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:32.312+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:32.433 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:32.424+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:32.435 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:32.428+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:32.737 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:32.730+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:32.759 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:32.753+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:32.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:32.879+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:33.015 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:33.009+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:33.218 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:33.211+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:33.331 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:33.326+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:33.450 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:33.444+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:33.453 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:33.448+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:33.748 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:33.741+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:33.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:33.742+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:33.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:33.881+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:34.009 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:34.002+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:34.192 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:34.186+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:34.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:34.302+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:34.446 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:34.439+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:34.450 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:34.444+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:34.708 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:34.698+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:34.770 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:34.765+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:34.873 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:34.867+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:35.009 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:35.002+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:35.196 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:35.190+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:35.356 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:35.351+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:35.455 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:35.447+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:35.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:35.450+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:35.723 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:35.716+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:35.741 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:35.735+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:35.867 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:35.861+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:35.973 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:35.966+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:36.184 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:36.178+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:36.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:36.383+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:36.415 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:36.409+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:36.417 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:36.412+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:36.761 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:36.754+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:36.765 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:36.758+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:36.892 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:36.886+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:36.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:36.951+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:37.205 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:37.197+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:37.373 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:37.367+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:37.379 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:37.373+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:37.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:37.378+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:37.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:37.730+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:37.801 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:37.795+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:37.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:37.878+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:37.943 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:37.938+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:38.179 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:38.172+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:38.332 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:38.327+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:38.336 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:38.331+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:38.421 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:38.415+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:38.701 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:38.693+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:38.794 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:38.787+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:38.938 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:38.929+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:38.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:38.952+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:39.184 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:39.176+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:39.345 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:39.340+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:39.349 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:39.344+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:39.414 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:39.410+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:39.731 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:39.724+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:39.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:39.796+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:39.992 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:39.980+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:40.011 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:40.004+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:40.213 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:40.205+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:40.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:40.394+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:40.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:40.398+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:40.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:40.455+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:40.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:40.780+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:40.822 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:40.815+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:40.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:40.977+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:41.025 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:41.011+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:41.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:41.181+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:41.403 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:41.397+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:41.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:41.399+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:41.433 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:41.429+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:41.794 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:41.786+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:41.797 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:41.789+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:41.958 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:41.952+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:42.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:42.020+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:42.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:42.191+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:42.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:42.397+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:42.452 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:42.447+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:42.456 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:42.451+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:42.766 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:42.758+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:42.847 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:42.838+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:42.973 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:42.966+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:42.978 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:02:42.971+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e249 v249) 2019-11-29T17:02:43.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:42.997+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:43.167 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:43.160+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:43.378 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:43.373+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:43.449 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:43.444+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:43.453 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:43.448+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:43.810 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:43.803+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:43.871 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:43.864+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:43.944 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:43.931+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:44.057 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:44.042+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:44.183 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:44.175+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:44.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:44.424+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:44.438 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:44.432+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:44.440 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:44.435+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:44.802 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:44.796+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:44.899 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:44.892+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:44.936 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:44.931+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:45.070 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:45.063+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:45.223 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:45.216+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:45.432 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:45.428+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:45.472 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:45.467+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:45.476 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:45.471+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:45.779 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:45.769+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:45.857 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:45.852+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:45.923 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:45.918+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:46.111 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:46.104+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:46.258 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:46.252+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:46.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:46.395+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:46.482 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:46.471+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:46.485 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:46.477+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:46.754 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:46.747+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:46.866 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:46.856+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:46.952 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:46.943+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:47.146 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:47.139+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:47.276 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:47.270+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:47.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:47.420+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:47.524 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:47.515+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:47.524 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:47.518+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:47.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:47.704+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:47.867 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:47.861+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:47.904 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:47.900+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:47.996 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:02:47.991+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e249 v249) 2019-11-29T17:02:48.185 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:48.179+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:48.330 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:48.324+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:48.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:48.463+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:48.567 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:48.561+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:48.571 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:48.567+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:48.721 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:48.707+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:48.889 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:48.879+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:48.897 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:48.889+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:49.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:49.141+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:49.296 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:49.289+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:49.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:49.491+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:49.582 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:49.574+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:49.584 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:49.578+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:49.703 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:49.697+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:49.893 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:49.887+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:49.932 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:49.926+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:50.128 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:50.121+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:50.345 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:50.339+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:50.524 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:50.519+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:50.562 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:50.555+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:50.567 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:50.563+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:50.668 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:50.662+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:50.889 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:50.884+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:50.890 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:50.886+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:51.179 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:51.173+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:51.317 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:51.301+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:51.523 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:51.518+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:51.527 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:51.520+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:51.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:51.555+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:51.687 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:51.681+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:51.858 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:51.851+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:51.874 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:51.869+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:52.153 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:52.147+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:52.329 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:52.323+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:52.551 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:52.546+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:52.551 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:52.546+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:52.556 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:52.551+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:52.735 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:52.729+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:52.897 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:52.891+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:52.898 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:52.892+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:53.021 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:02:53.005+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e249 v249) 2019-11-29T17:02:53.189 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:53.181+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:53.311 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:53.304+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:53.555 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:53.547+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:53.556 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:53.551+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:53.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:53.556+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:53.745 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:53.739+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:53.925 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:53.920+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:53.941 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:53.936+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:54.226 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:54.220+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:54.335 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:54.329+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:54.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:54.531+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:54.542 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:54.535+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:54.544 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:54.539+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:54.726 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:54.720+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:54.943 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:54.937+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:54.950 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:54.944+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:55.209 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:55.203+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:55.321 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:55.316+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:55.568 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:55.560+0000 1694c700 -1 osd.3 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.624733+0000 front 2019-11-29T17:01:29.360603+0000 (oldest deadline 2019-11-29T17:02:55.072476+0000) 2019-11-29T17:02:55.568 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:55.563+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:55.584 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:55.577+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:55.585 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:55.581+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:55.789 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:55.781+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:55.949 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:55.943+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:56.002 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:55.995+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:56.251 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:56.245+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:56.289 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:56.282+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:56.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:56.518+0000 1694c700 -1 osd.3 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.624733+0000 front 2019-11-29T17:01:29.360603+0000 (oldest deadline 2019-11-29T17:02:55.072476+0000) 2019-11-29T17:02:56.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:56.521+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:56.612 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:56.603+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:56.613 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:56.607+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:56.836 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:56.830+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:56.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:56.960+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:57.017 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:57.011+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:57.237 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:02:57.231+0000 1694c700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:27.792161+0000 front 2019-11-29T17:01:27.566967+0000 (oldest deadline 2019-11-29T17:02:24.350892+0000) 2019-11-29T17:02:57.310 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:02:57.300+0000 1694c700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:26.931637+0000 front 2019-11-29T17:01:26.922502+0000 (oldest deadline 2019-11-29T17:02:11.008177+0000) 2019-11-29T17:02:57.546 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:57.540+0000 1694c700 -1 osd.3 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.624733+0000 front 2019-11-29T17:01:29.360603+0000 (oldest deadline 2019-11-29T17:02:55.072476+0000) 2019-11-29T17:02:57.547 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:57.542+0000 1694c700 -1 osd.3 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:57.612 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:57.603+0000 1694c700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:01:43.881969+0000 (oldest deadline 2019-11-29T17:02:23.881969+0000) 2019-11-29T17:02:57.614 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:02:57.608+0000 1694c700 -1 osd.0 249 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-11-29T17:02:57.839 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:02:57.833+0000 1694c700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.660582+0000 front 2019-11-29T17:01:28.662136+0000 (oldest deadline 2019-11-29T17:02:13.088624+0000) 2019-11-29T17:02:57.939 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:02:57.932+0000 1694c700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:28.562094+0000 front 2019-11-29T17:01:28.449069+0000 (oldest deadline 2019-11-29T17:02:12.470692+0000) 2019-11-29T17:02:58.033 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:02:58.024+0000 1694c700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:01:29.631932+0000 front 2019-11-29T17:01:29.696030+0000 (oldest deadline 2019-11-29T17:02:13.579434+0000) 2019-11-29T17:02:58.070 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:02:58.065+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e249 v249) 2019-11-29T17:02:58.784 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:58.762+0000 1694c700 -1 osd.3 250 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101426 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:02:59.867 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:02:59.848+0000 1694c700 -1 osd.3 251 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101457 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 1472469456 gen 1] snapc 0=[] ondisk+write+known_if_redirected e249) 2019-11-29T17:03:11.854 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:50:24.743 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:11.863 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:50:24.751 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:11.870 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:50:24.759 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:11.881 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:50:24.769 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:11.889 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:50:24.664 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:11.898 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:50:24.672 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:12.107 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:50:24.996 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:12.183 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:50:25.069 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:12.183 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:50:24.954 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:03:53.477 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:03:53.471+0000 1694c700 -1 osd.0 254 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=39 ondisk+retry+read+known_if_redirected e253) 2019-11-29T17:03:54.809 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:03:54.682+0000 1694c700 -1 osd.0 254 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=39 ondisk+retry+read+known_if_redirected e253) 2019-11-29T17:03:55.849 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:03:55.825+0000 1694c700 -1 osd.0 254 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=39 ondisk+retry+read+known_if_redirected e253) 2019-11-29T17:04:46.595 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:46.589+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:47.278 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:47.271+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:47.578 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:47.571+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:47.647 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:47.641+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:48.330 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:48.323+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:48.568 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:48.562+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:48.689 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:48.683+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:48.999 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:48.990+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:49.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:48.999+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:49.016 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:49.007+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:49.342 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:49.336+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:49.348 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:49.342+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:49.546 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:49.539+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:49.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:49.721+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:49.979 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:49.973+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:50.004 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:49.999+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:50.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:50.011+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:50.326 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:50.317+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:50.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:50.339+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:50.532 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:50.526+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:50.698 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:50.691+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:50.989 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:50.982+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:50.996 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:50.989+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:51.011 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:51.005+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:51.334 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:51.328+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:51.344 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:51.339+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:51.552 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:51.546+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:51.656 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:51.649+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:52.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:52.011+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:52.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:52.035+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:52.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:52.047+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:52.310 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:52.302+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:52.316 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:52.310+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:52.557 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:52.550+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:52.619 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:52.612+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:53.002 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:52.995+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:53.021 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:53.016+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:53.073 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:53.066+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:53.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:53.307+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:53.358 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:53.351+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:53.514 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:53.508+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:53.636 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:53.629+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:53.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:53.985+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:54.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:53.999+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:54.036 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:54.029+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:54.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:54.276+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:54.397 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:54.391+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:54.483 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:54.477+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:54.623 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:54.617+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:54.960 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:54.945+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:54.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:54.966+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:55.067 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:55.060+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:55.285 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:55.279+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:55.422 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:55.416+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:55.439 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:55.433+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:55.616 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:55.610+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:55.990 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:55.983+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:55.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:55.988+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:56.023 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:56.017+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:56.299 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:56.293+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:56.382 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:56.376+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:56.437 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:56.432+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:56.670 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:56.664+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:56.968 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:56.962+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:56.997 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:56.990+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:57.006 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:57.000+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:57.327 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:57.321+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:57.426 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:57.421+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:57.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:57.422+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:57.683 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:57.676+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:57.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:57.971+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:57.998 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:57.991+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:58.036 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:58.030+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:58.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:58.289+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:58.403 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:58.398+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:58.406 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:58.400+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:58.677 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:58.671+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:59.011 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:59.006+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:04:59.051 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:04:59.045+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:04:59.074 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:04:59.068+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:04:59.264 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:04:59.258+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:04:59.452 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:04:59.446+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:04:59.456 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:04:59.447+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:04:59.683 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:04:59.676+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:04:59.992 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:04:59.986+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:00.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:00.008+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:00.075 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:00.068+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:00.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:00.253+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:00.465 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:00.459+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:00.504 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:00.499+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:00.725 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:00.719+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:01.003 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:00.997+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:01.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:01.019+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:01.084 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:01.079+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:01.222 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:01.216+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:01.471 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:01.464+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:01.540 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:01.534+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:01.719 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:01.713+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:01.958 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:01.953+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:02.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:02.021+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:02.069 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:02.062+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:02.264 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:02.259+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:02.430 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:02.424+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:02.501 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:02.495+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:02.771 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:02.764+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:03.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:03.004+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:03.043 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:03.038+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:03.120 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:03.113+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:03.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:03.215+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:03.430 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:03.425+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:03.500 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:03.493+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:03.769 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:03.763+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:04.054 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:04.049+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:04.072 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:04.066+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:04.131 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:04.127+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:04.194 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:04.189+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:04.423 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:04.418+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:04.493 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:04.486+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:04.804 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:04.797+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:05.074 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:05.069+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:05.120 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:05.114+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:05.162 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:05.155+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:05.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:05.237+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:05.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:05.452+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:05.466 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:05.460+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:05.813 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:05.806+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:06.068 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:06.063+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:06.166 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:06.159+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:06.197 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:06.190+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:06.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:06.236+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:06.469 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:06.463+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:06.510 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:06.504+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:06.801 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:06.795+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:07.035 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:07.030+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:07.175 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:07.169+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:07.199 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:07.193+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:07.213 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:07.206+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:07.456 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:07.450+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:07.466 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:07.460+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:07.840 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:07.834+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:07.996 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:07.991+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:08.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:08.186+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:08.225 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:08.219+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:08.237 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:08.230+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:08.441 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:08.434+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:08.469 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:08.462+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:08.891 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:08.884+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:08.977 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:08.970+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:09.157 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:09.149+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:09.245 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:09.239+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:09.281 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:09.274+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:09.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:09.421+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:09.473 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:09.467+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:09.937 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:09.929+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:10.001 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:09.995+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:10.171 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:10.163+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:10.239 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:10.233+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:10.263 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:10.255+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:10.409 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:10.402+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:10.503 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:10.496+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:10.957 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:10.948+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:11.020 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:11.016+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:11.182 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:11.175+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:11.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:11.196+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:11.303 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:11.296+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:11.367 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:11.360+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:11.489 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:11.484+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:11.916 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:11.909+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:11.996 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:11.991+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:12.174 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:12.163+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:12.175 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:12.168+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:12.316 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:12.309+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:12.362 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:12.355+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:12.508 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:12.503+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:12.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:12.912+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:12.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:12.969+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:13.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:13.148+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:13.176 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:13.168+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:13.323 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:13.315+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:13.355 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:13.347+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:13.483 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:13.477+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:13.960 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:13.952+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:13.992 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:13.987+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:14.156 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:14.148+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:14.162 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:14.156+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:14.316 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:14.308+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:14.370 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:14.362+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:14.526 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:14.520+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:14.966 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:14.955+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:15.016 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:15.010+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:15.156 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:15.150+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:15.183 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:15.175+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:15.294 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:15.286+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:15.419 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:15.409+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:15.503 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:15.497+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:15.955 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:15.948+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:16.020 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:16.015+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:16.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:16.183+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:16.206 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:16.200+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:16.277 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:16.270+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:16.379 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:16.372+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:16.557 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:16.551+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:16.979 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:16.971+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:17.046 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:17.041+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:17.148 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:17.141+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:17.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:17.216+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:17.261 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:17.254+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:17.364 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:17.357+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:17.587 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:17.581+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:18.008 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:05:18.001+0000 1694c700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.522638+0000 front 2019-11-29T17:04:04.449343+0000 (oldest deadline 2019-11-29T17:04:46.392074+0000) 2019-11-29T17:05:18.093 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:18.088+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:18.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:05:18.098+0000 1694c700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.202604+0000 front 2019-11-29T17:04:04.200524+0000 (oldest deadline 2019-11-29T17:04:48.120607+0000) 2019-11-29T17:05:18.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:18.260+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:05:18.266 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:05:18.259+0000 1694c700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.216723+0000 front 2019-11-29T17:04:04.218295+0000 (oldest deadline 2019-11-29T17:04:48.723532+0000) 2019-11-29T17:05:18.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:05:18.387+0000 1694c700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.185946+0000 front 2019-11-29T17:04:02.995388+0000 (oldest deadline 2019-11-29T17:04:47.541098+0000) 2019-11-29T17:05:18.584 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:05:18.578+0000 1694c700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:04.169571+0000 front 2019-11-29T17:04:04.040271+0000 (oldest deadline 2019-11-29T17:04:48.629572+0000) 2019-11-29T17:05:18.697 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:05:18.689+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e254 v254) 2019-11-29T17:05:18.883 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:52:31.771 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:18.894 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:52:31.783 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:18.904 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:52:31.792 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:18.918 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:52:31.806 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:18.923 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:52:31.697 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:18.927 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:52:31.702 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:19.020 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:52:31.909 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:19.043 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:52:31.818 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:19.047 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:52:31.935 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:05:19.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:05:19.128+0000 1694c700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:03.139303+0000 front 2019-11-29T17:04:03.283964+0000 (oldest deadline 2019-11-29T17:04:48.670513+0000) 2019-11-29T17:05:19.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:05:19.286+0000 1694c700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:04:01.994592+0000 front 2019-11-29T17:04:01.996878+0000 (oldest deadline 2019-11-29T17:04:46.875542+0000) 2019-11-29T17:06:16.428 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:16.420+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:17.414 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:17.405+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:17.414 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:17.406+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:18.385 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:18.378+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:18.440 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:18.434+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:18.620 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:06:18.356+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414848672 err (107) Transport endpoint is not connected 2019-11-29T17:06:19.091 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:19.079+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:19.290 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:19.282+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:19.418 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:19.413+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:19.433 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:19.424+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:20.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:20.062+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:20.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:20.241+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:20.384 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:20.376+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:20.396 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:20.388+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:20.659 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:20.652+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:20.766 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:20.759+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:21.110 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:21.104+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:21.236 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:21.230+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:21.419 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:21.413+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:21.424 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:21.417+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:21.642 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:21.613+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:21.774 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:21.768+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:22.093 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:22.087+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:22.264 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:22.257+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:22.439 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:22.433+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:22.457 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:22.451+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:22.623 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:22.615+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:22.755 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:22.749+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:23.049 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:23.043+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:23.263 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:23.256+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:23.404 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:23.398+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:23.509 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:23.503+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:23.631 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:23.624+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:23.775 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:23.769+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:24.032 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:24.026+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:24.272 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:24.267+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:24.410 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:24.403+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:24.534 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:24.523+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:24.633 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:24.629+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:24.750 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:24.743+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:25.010 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:25.003+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:25.289 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:25.283+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:25.767 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:25.429+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:25.767 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:25.571+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:25.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:25.701+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:25.767 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:25.655+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:25.767 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:25.665+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:26.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:26.046+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:26.302 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:26.297+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:26.437 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:26.430+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:26.618 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:26.613+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:26.624 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:26.617+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:26.679 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:26.675+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:26.758 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:26.750+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:27.040 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:27.033+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:27.309 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:27.303+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:27.511 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:27.484+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:27.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:27.607+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:27.641 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:27.635+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:27.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:27.639+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:27.744 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:27.737+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:28.092 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:28.085+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:28.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:28.344+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:28.524 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:28.517+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:28.584 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:28.576+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:28.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:28.640+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:28.691 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:28.686+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:28.724 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:28.718+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:29.143 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:29.136+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:29.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:29.339+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:29.542 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:29.533+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:29.561 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:29.554+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:29.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:29.637+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:29.646 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:29.642+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:29.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:29.682+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:29.715 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:29.708+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:30.158 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:30.152+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:30.347 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:30.341+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:30.541 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:30.533+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:30.566 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:30.558+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:30.664 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:30.657+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:30.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:30.666+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:30.684 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:30.677+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:30.693 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:30.688+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:31.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:31.187+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:31.397 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:31.391+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:31.575 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:31.568+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:31.578 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:31.572+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:31.636 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:31.630+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:31.640 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:31.635+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:31.697 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:31.690+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:31.734 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:31.727+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:32.169 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:32.162+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:32.400 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:32.394+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:32.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:32.549+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:32.599 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:32.591+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:32.608 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:32.596+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:32.613 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:32.605+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:32.703 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:32.696+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:32.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:32.721+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:33.151 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:33.141+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:33.413 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:33.407+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:33.578 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:33.568+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:33.580 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:33.574+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:33.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:33.646+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:33.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:33.650+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:33.683 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:33.677+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:33.744 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:33.738+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:33.947 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:06:33.902+0000 20e22700 -1 RGWWatcher::handle_error cookie 5292892000 err (107) Transport endpoint is not connected 2019-11-29T17:06:34.205 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:34.193+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:34.409 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:34.403+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:34.598 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:34.591+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:34.632 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:34.626+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:34.640 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:34.635+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:34.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:34.639+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:34.683 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:34.676+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:34.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:34.701+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:35.182 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:35.176+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:35.399 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:35.392+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:35.603 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:35.596+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:35.603 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:35.598+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:35.643 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:35.637+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:35.661 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:35.655+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:35.671 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:35.665+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:35.745 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:35.739+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:36.154 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:36.148+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:36.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:36.370+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:36.599 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:36.592+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:36.601 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:36.596+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:36.663 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:36.658+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:36.691 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:36.684+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:36.706 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:36.698+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:36.715 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:36.709+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:37.116 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:37.110+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:37.347 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:37.340+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:37.621 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:37.615+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:37.635 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:37.631+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:37.637 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:37.633+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:37.687 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:37.681+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:37.731 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:37.725+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:37.742 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:37.735+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:38.132 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:38.125+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:38.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:38.380+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:38.619 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:38.613+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:38.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:38.616+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:38.651 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:38.645+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:38.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:38.647+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:38.667 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:38.661+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:38.729 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:38.724+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:38.730 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:38.723+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:39.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:39.099+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:39.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:39.342+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:39.631 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:39.624+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:39.631 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:39.626+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:39.648 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:39.642+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:39.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:39.682+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:39.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:39.687+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:39.731 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:39.724+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:39.777 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:39.767+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:40.118 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:40.112+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:40.352 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:40.345+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:40.615 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:40.609+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:40.616 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:40.611+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:40.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:40.683+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:40.689 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:40.684+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:40.703 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:40.697+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:40.707 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:40.700+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:40.746 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:40.740+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:41.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:41.080+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:41.310 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:41.304+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:41.633 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:41.626+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:41.634 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:41.628+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:41.663 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:41.656+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:41.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:41.659+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:41.664 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:41.657+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:41.683 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:41.677+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:41.723 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:41.716+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:42.071 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:42.065+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:42.358 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:42.353+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:42.640 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:42.635+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:42.640 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:42.636+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:42.640 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:42.634+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:42.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:42.660+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:42.669 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:42.664+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:42.726 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:42.720+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:42.727 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:42.719+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:43.075 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:43.069+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:43.323 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:43.316+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:43.650 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:43.643+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:43.654 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:43.647+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:43.655 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:43.650+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:43.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:43.650+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:43.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:43.652+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:43.696 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:43.690+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:43.697 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:43.691+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:44.132 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:44.126+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:44.284 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:44.279+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:44.621 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:44.610+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:44.625 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:44.616+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:44.689 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:44.684+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:44.690 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:44.682+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:44.694 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:44.687+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:44.742 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:44.730+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:44.757 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:44.740+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:45.120 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:45.105+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:45.319 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:45.313+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:45.616 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:45.609+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:45.618 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:45.612+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:45.665 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:45.659+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:45.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:45.661+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:45.667 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:45.662+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:45.768 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:45.761+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:45.786 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:45.779+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:46.144 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:46.138+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:46.299 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:46.294+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:46.587 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:46.580+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:46.588 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:46.583+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:46.622 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:46.616+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:46.625 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:46.621+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:46.700 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:46.695+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:46.804 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:46.798+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:46.812 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:46.806+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:47.177 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:47.164+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:47.300 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:47.295+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:47.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:47.582+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:47.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:47.586+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:47.614 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:47.608+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:47.616 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:47.611+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:47.711 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:47.704+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:47.811 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:47.804+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:47.853 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:47.847+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:48.178 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:48.171+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:48.257 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:48.252+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:48.576 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:48.572+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:48.577 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:48.574+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:48.611 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:48.606+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:48.615 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:48.611+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:48.707 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:48.701+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:48.790 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:48.783+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:48.874 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:48.867+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:49.148 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:49.141+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:49.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:49.275+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:49.547 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:49.541+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:49.551 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:49.546+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:49.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:49.639+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:49.649 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:49.644+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:49.688 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:49.681+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:49.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:49.782+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:49.859 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:49.853+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:50.188 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:50.182+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:50.283 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:50.276+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:50.561 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:50.553+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:50.561 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:50.555+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:50.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:50.684+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:50.694 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:50.688+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:50.735 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:50.729+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:50.775 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:50.769+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:50.871 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:50.865+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:51.161 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:51.154+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:51.292 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:51.285+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:51.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:51.532+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:51.539 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:51.533+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:51.703 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:51.696+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:51.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:51.714+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:51.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:51.717+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:51.741 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:51.735+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:51.915 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:51.908+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:52.210 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:52.204+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:52.327 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:52.321+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:52.500 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:52.493+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:52.501 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:52.496+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:52.678 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:52.672+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:52.754 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:52.748+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:52.758 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:52.751+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:52.769 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:52.762+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:52.938 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:52.932+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:53.259 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:53.253+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:53.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:53.373+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:53.502 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:53.488+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:53.503 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:53.491+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:53.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:53.721+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:53.786 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:53.779+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:53.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:53.782+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:53.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:53.782+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:53.904 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:53.897+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:53.959 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:06:53.954+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e260 v260) 2019-11-29T17:06:54.252 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:54.246+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:54.371 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:54.366+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:54.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:54.498+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:54.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:54.501+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:54.739 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:54.732+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:54.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:54.740+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:54.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:54.744+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:54.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:54.760+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:54.957 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:54.944+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:55.232 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:55.226+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:55.405 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:55.399+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:55.460 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:55.454+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:55.465 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:55.458+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:55.719 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:55.713+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:55.723 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:55.717+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:55.728 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:55.721+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:55.763 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:55.757+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:55.910 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:55.904+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:56.206 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:56.200+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:56.441 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:56.435+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:56.495 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:56.484+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:56.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:56.485+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:56.683 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:56.677+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:56.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:56.681+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:56.728 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:56.722+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:56.744 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:56.738+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:56.912 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:56.898+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:57.207 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:57.200+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:57.491 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:57.484+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:57.534 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:57.527+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:57.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:57.531+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:57.734 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:57.723+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:57.734 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:57.726+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:57.737 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:57.731+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:57.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:57.760+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:57.892 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:57.886+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:58.244 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:06:58.237+0000 1694c700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.159318+0000 front 2019-11-29T17:05:34.197687+0000 (oldest deadline 2019-11-29T17:06:18.173440+0000) 2019-11-29T17:06:58.456 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:06:58.450+0000 1694c700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:31.214518+0000 front 2019-11-29T17:05:31.217293+0000 (oldest deadline 2019-11-29T17:06:18.740033+0000) 2019-11-29T17:06:58.544 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:58.537+0000 1694c700 -1 osd.3 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.168576+0000 front 2019-11-29T17:05:35.041690+0000 (oldest deadline 2019-11-29T17:06:19.701823+0000) 2019-11-29T17:06:58.544 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:58.539+0000 1694c700 -1 osd.3 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:06:58.764 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:06:58.758+0000 1694c700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:34.804784+0000 front 2019-11-29T17:05:34.768549+0000 (oldest deadline 2019-11-29T17:06:20.039875+0000) 2019-11-29T17:06:58.766 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:58.758+0000 1694c700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:05:49.158407+0000 (oldest deadline 2019-11-29T17:06:29.158407+0000) 2019-11-29T17:06:58.767 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:06:58.761+0000 1694c700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:100964 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-11-29T17:06:58.800 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:06:58.794+0000 1694c700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.230818+0000 front 2019-11-29T17:05:35.229988+0000 (oldest deadline 2019-11-29T17:06:16.098981+0000) 2019-11-29T17:06:58.907 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:06:58.900+0000 1694c700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:05:35.008801+0000 front 2019-11-29T17:05:34.961513+0000 (oldest deadline 2019-11-29T17:06:17.089988+0000) 2019-11-29T17:06:59.005 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:06:58.999+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e260 v260) 2019-11-29T17:06:59.629 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:06:59.578+0000 1694c700 -1 osd.3 261 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:07:00.580 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:07:00.572+0000 1694c700 -1 osd.3 262 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101830 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5292892000] snapc 0=[] ondisk+write+known_if_redirected e260) 2019-11-29T17:07:13.317 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:54:26.205 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.326 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:54:26.215 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.342 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:54:26.230 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.348 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:54:26.122 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.350 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:54:26.238 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.366 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:54:26.140 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.512 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:54:26.401 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.554 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:54:26.442 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:07:13.556 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:54:26.330 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:08:52.090 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:52.084+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:52.367 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:52.359+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:52.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:52.484+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:53.046 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:53.039+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:53.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:53.289+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:53.341 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:53.336+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:53.522 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:53.507+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:54.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:54.032+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:54.229 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:54.223+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:54.303 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:54.297+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:54.371 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:54.364+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:54.509 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:54.502+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:54.923 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:54.915+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:55.068 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:55.062+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:55.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:55.262+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:55.323 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:55.317+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:55.405 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:55.398+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:55.516 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:55.508+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:55.853 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:08:55.844+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:08:55.927 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:55.922+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:56.031 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:56.024+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:56.224 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:56.219+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:56.286 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:56.281+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:56.426 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:56.419+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:56.478 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:56.470+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:56.824 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:08:56.816+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:08:56.890 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:56.885+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:57.083 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:57.076+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:57.252 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:57.245+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:57.269 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:57.262+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:57.414 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:57.407+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:57.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:57.484+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:57.848 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:57.843+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:57.884 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:08:57.866+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:08:58.124 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:58.116+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:58.232 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:58.226+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:58.235 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:58.228+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:58.391 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:58.379+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:58.542 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:58.534+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:58.815 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:58.810+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:58.879 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:08:58.872+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:08:59.087 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:08:59.074+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:08:59.190 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:08:59.185+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:08:59.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:08:59.186+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:08:59.355 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:08:59.348+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:08:59.566 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:08:59.558+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:08:59.851 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:08:59.845+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:08:59.887 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:08:59.879+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:00.037 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:00.029+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:00.157 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:00.151+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:00.196 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:00.189+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:00.403 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:00.395+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:00.573 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:00.565+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:00.809 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:00.803+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:00.857 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:00.849+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:01.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:01.063+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:01.146 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:01.140+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:01.170 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:01.163+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:01.438 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:01.431+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:01.586 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:01.579+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:01.780 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:01.774+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:01.896 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:01.888+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:02.106 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:02.098+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:02.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:02.142+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:02.164 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:02.158+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:02.404 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:02.396+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:02.549 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:02.541+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:02.744 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:02.738+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:02.887 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:02.879+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:03.108 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:03.103+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:03.122 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:03.115+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:03.196 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:03.190+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:03.424 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:03.417+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:03.537 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:03.530+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:03.723 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:03.716+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:03.852 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:03.841+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:04.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:04.094+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:04.153 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:04.145+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:04.205 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:04.200+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:04.466 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:04.459+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:04.507 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:04.500+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:04.683 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:04.677+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:04.859 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:04.852+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:05.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:05.094+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:05.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:05.140+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:05.207 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:05.201+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:05.470 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:05.463+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:05.491 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:05.485+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:05.680 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:05.674+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:05.882 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:05.876+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:06.073 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:06.068+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:06.109 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:06.102+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:06.180 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:06.175+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:06.509 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:06.503+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:06.540 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:06.534+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:06.647 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:06.641+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:06.852 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:06.845+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:07.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:07.059+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:07.079 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:07.072+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:07.217 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:07.210+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:07.528 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:07.521+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:07.532 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:07.524+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:07.668 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:07.662+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:07.906 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:07.899+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:08.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:08.051+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:08.073 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:08.068+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:08.208 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:08.201+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:08.516 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:08.505+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:08.572 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:08.566+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:08.635 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:08.628+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:08.925 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:08.919+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:09.014 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:09.008+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:09.122 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:09.117+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:09.166 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:09.159+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:09.557 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:09.550+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:09.594 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:09.588+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:09.622 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:09.616+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:09.960 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:09.954+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:10.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:10.006+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:10.153 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:10.147+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:10.179 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:10.174+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:10.577 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:10.571+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:10.643 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:10.635+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:10.647 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:10.639+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:10.926 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:10.920+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:11.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:11.013+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:11.144 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:11.137+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:11.194 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:11.187+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:11.537 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:11.530+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:11.653 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:11.647+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:11.694 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:11.686+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:11.893 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:11.886+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:12.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:12.006+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:12.172 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:12.166+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:12.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:12.183+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:12.592 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:12.583+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:12.652 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:12.645+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:12.680 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:12.674+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:12.938 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:12.930+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:12.979 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:12.974+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:13.135 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:13.128+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:13.209 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:13.202+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:13.584 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:13.578+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:13.642 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:13.636+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:13.644 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:13.638+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:13.900 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:13.893+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:13.949 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:13.942+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:14.122 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:14.115+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:14.202 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:14.195+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:14.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:14.603+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:14.643 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:14.637+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:14.645 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:14.638+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:14.875 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:14.869+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:14.998 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:14.991+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:15.078 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:15.072+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:15.213 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:15.206+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:15.601 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:15.594+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:15.617 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:15.610+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:15.690 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:15.683+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:15.881 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:15.873+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:16.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:16.015+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:16.122 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:16.115+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:16.205 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:16.198+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:16.589 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:16.583+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:16.640 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:16.634+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:16.677 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:16.671+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:16.916 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:16.910+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:17.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:17.059+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:17.111 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:17.104+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:17.196 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:17.189+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:17.617 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:17.611+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:17.645 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:17.639+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:17.659 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:17.653+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:17.908 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:17.901+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:18.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:18.097+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:18.107 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:18.100+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:18.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:18.184+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:18.606 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:18.599+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:18.626 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:18.620+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:18.687 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:18.680+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:18.939 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:18.929+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:19.108 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:19.102+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:19.140 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:19.133+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:19.190 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:19.184+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:19.638 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:19.631+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:19.675 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:19.668+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:19.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:19.725+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:19.911 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:19.905+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:20.128 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:20.119+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:20.160 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:20.147+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:20.180 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:20.173+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:20.651 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:20.645+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:20.726 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:20.718+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:20.748 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:20.741+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:20.910 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:20.904+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:21.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:21.137+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:21.152 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:21.145+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:21.231 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:21.224+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:21.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:21.662+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:21.716 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:21.710+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:21.798 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:21.791+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:21.966 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:21.959+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:22.173 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:22.166+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:22.191 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:22.184+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:22.192 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:22.186+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:22.677 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:22.671+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:22.684 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:22.678+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:22.849 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:22.843+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:22.977 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:22.971+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:23.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:23.143+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:23.199 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:23.193+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:23.238 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:23.231+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:23.652 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:23.645+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:23.697 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:23.690+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:23.818 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:23.811+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:24.007 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:24.001+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:24.159 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:24.152+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:24.194 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:24.186+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:24.224 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:24.217+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:24.543 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:09:24.537+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e266 v266) 2019-11-29T17:09:24.656 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:24.651+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:24.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:24.662+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:24.779 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:24.773+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:25.014 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:24.998+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:25.148 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:25.142+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:25.173 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:25.166+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:25.233 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:25.227+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:25.694 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:25.686+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:25.700 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:25.694+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:25.820 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:25.813+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:26.024 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:26.018+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:26.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:26.098+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:26.135 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:26.130+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:26.201 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:26.196+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:26.728 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:26.722+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:26.738 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:26.731+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:26.823 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:26.816+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:27.056 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:27.049+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:27.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:27.097+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:27.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:27.127+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:27.245 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:27.238+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:27.697 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:27.691+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:27.783 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:27.777+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:27.840 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:27.833+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:28.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:28.062+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:28.088 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:28.082+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:28.184 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:28.178+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:28.274 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:28.267+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:28.659 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:28.654+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:09:28.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:09:28.782+0000 1694c700 -1 osd.5 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:11.901163+0000 front 2019-11-29T17:08:10.783016+0000 (oldest deadline 2019-11-29T17:08:51.513627+0000) 2019-11-29T17:09:28.851 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:09:28.844+0000 1694c700 -1 osd.4 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:07.600224+0000 front 2019-11-29T17:08:07.604518+0000 (oldest deadline 2019-11-29T17:08:51.667017+0000) 2019-11-29T17:09:29.088 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:29.082+0000 1694c700 -1 osd.6 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:09.495250+0000 front 2019-11-29T17:08:09.621897+0000 (oldest deadline 2019-11-29T17:08:55.329792+0000) 2019-11-29T17:09:29.094 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:09:29.087+0000 1694c700 -1 osd.3 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.031145+0000 front 2019-11-29T17:08:08.028124+0000 (oldest deadline 2019-11-29T17:08:53.907621+0000) 2019-11-29T17:09:29.227 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:09:29.221+0000 1694c700 -1 osd.2 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:08.836897+0000 front 2019-11-29T17:08:08.849708+0000 (oldest deadline 2019-11-29T17:08:52.603412+0000) 2019-11-29T17:09:29.302 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:09:29.296+0000 1694c700 -1 osd.7 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:06.103733+0000 front 2019-11-29T17:08:06.101379+0000 (oldest deadline 2019-11-29T17:08:51.785017+0000) 2019-11-29T17:09:29.611 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:09:29.604+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e266 v266) 2019-11-29T17:09:29.710 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:09:29.702+0000 1694c700 -1 osd.1 266 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:08:12.035858+0000 front 2019-11-29T17:08:12.265976+0000 (oldest deadline 2019-11-29T17:08:54.172454+0000) 2019-11-29T17:10:00.006 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:09:59.996+0000 1694c700 -1 osd.6 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102190 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821264656 gen 88] snapc 0=[] ondisk+write+known_if_redirected e267) 2019-11-29T17:10:04.665 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:04.641+0000 38279700 -1 osd.6 270 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:04.895 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:04.829+0000 34271700 -1 osd.3 270 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:04.935 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:04.921+0000 34271700 -1 osd.3 270 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:04.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:04.897+0000 3b27f700 -1 osd.6 270 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:08.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:08.030+0000 11cac700 -1 osd.3 271 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:08.201 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:08.191+0000 11cac700 -1 osd.6 271 build_incremental_map_msg missing incremental map 270 2019-11-29T17:10:27.370 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:27.363+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:28.096 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:28.091+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:28.408 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:28.402+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:28.924 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:10:28.914+0000 20e22700 -1 RGWWatcher::handle_error cookie 5415392464 err (107) Transport endpoint is not connected 2019-11-29T17:10:29.094 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:29.089+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:29.411 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:29.403+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:30.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:30.011+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:30.075 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:30.070+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:30.224 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:30.216+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:30.445 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:30.437+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:31.008 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:31.002+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:31.090 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:31.084+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:31.266 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:31.260+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:31.331 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:31.325+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:31.440 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:31.433+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:31.992 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:31.985+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:32.064 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:32.056+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:32.302 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:32.296+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:32.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:32.336+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:32.462 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:32.456+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:32.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:32.979+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:33.021 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:33.016+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:33.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:33.058+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:33.336 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:33.329+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:33.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:33.352+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:33.487 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:33.477+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:33.996 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:33.989+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:34.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:34.045+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:34.083 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:34.077+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:34.375 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:34.370+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:34.387 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:34.381+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:34.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:34.463+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:34.521 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:34.513+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:34.981 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:34.969+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:35.043 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:35.036+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:35.093 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:35.087+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:35.369 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:35.363+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:35.423 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:35.417+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:35.483 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:35.479+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:35.555 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:35.549+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:36.008 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:36.001+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:36.039 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:36.033+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:36.124 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:36.120+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:36.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:36.374+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:36.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:36.413+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:36.499 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:36.494+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:36.586 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:36.579+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:37.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:37.003+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:37.064 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:37.058+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:37.139 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:37.133+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:37.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:37.372+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:37.472 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:37.466+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:37.492 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:37.487+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:37.633 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:37.627+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:37.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:37.969+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:38.116 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:38.111+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:38.117 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:38.111+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:38.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:38.386+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:38.502 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:38.496+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:38.503 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:38.498+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:38.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:38.631+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:38.981 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:38.974+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:39.140 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:39.133+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:39.146 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:39.140+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:39.426 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:39.420+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:39.482 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:39.475+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:39.543 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:39.537+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:39.639 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:39.633+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:39.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:39.957+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:40.113 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:40.107+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:40.177 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:40.170+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:40.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:40.416+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:40.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:40.486+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:40.560 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:40.555+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:40.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:40.633+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:40.978 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:40.972+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:41.139 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:41.128+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:41.193 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:41.187+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:41.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:41.381+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:41.514 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:41.508+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:41.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:41.600+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:41.610 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:41.604+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:41.616 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:41.611+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:42.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:42.006+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:42.126 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:42.119+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:42.154 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:42.146+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:42.433 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:42.414+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:42.557 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:42.550+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:42.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:42.566+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:42.575 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:42.569+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:42.606 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:42.592+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:42.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:42.991+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:43.079 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:43.072+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:43.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:43.183+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:43.450 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:43.444+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:43.512 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:43.505+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:43.611 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:43.604+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:43.612 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:43.606+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:43.642 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:43.635+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:44.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:43.994+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:44.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:44.057+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:44.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:44.182+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:44.428 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:44.420+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:44.497 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:44.490+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:44.568 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:44.562+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:44.569 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:44.564+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:44.627 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:44.621+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:45.050 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:45.044+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:45.098 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:45.080+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:45.169 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:45.161+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:45.397 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:45.391+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:45.472 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:45.466+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:45.592 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:45.584+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:45.592 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:45.587+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:45.634 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:45.628+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:46.062 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:46.056+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:46.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:46.057+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:46.182 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:46.174+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:46.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:46.384+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:46.456 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:46.444+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:46.613 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:46.607+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:46.625 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:46.620+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:46.631 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:46.625+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:47.021 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:47.015+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:47.057 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:47.050+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:47.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:47.186+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:47.432 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:47.426+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:47.465 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:47.458+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:47.626 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:47.619+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:47.646 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:47.640+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:47.649 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:47.643+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:47.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:47.979+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:48.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:48.069+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:48.190 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:48.183+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:48.453 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:48.442+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:48.486 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:48.476+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:48.624 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:48.618+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:48.628 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:48.622+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:48.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:48.632+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:48.987 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:48.980+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:49.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:49.047+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:49.169 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:49.161+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:49.443 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:49.430+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:49.465 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:49.454+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:49.675 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:49.667+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:49.677 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:49.671+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:49.688 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:49.681+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:49.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:49.979+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:50.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:50.007+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:50.165 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:50.158+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:50.400 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:50.393+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:50.447 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:50.439+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:50.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:50.626+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:50.635 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:50.629+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:50.726 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:50.719+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:50.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:50.950+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:51.003 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:50.997+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:51.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:51.157+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:51.448 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:51.441+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:51.503 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:51.492+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:51.627 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:51.619+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:51.628 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:51.622+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:51.721 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:51.714+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:51.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:51.972+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:52.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:52.011+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:52.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:52.135+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:52.492 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:52.485+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:52.504 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:52.497+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:52.681 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:52.673+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:52.681 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:52.676+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:52.694 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:52.687+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:52.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:52.949+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:53.033 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:53.025+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:53.194 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:53.186+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:53.461 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:53.454+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:53.475 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:53.469+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:53.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:53.685+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:53.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:53.687+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:53.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:53.739+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:53.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:53.967+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:54.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:54.019+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:54.182 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:54.174+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:54.466 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:54.459+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:54.515 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:54.507+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:54.724 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:54.714+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:54.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:54.718+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:54.794 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:54.788+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:54.905 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:10:54.896+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414896208 err (107) Transport endpoint is not connected 2019-11-29T17:10:54.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:54.966+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:55.021 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:55.014+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:55.185 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:55.177+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:55.447 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:55.414+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:55.568 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:55.525+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:55.723 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:55.717+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:55.725 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:55.720+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:55.839 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:55.831+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:55.955 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:55.948+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:56.059 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:56.052+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:56.213 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:56.206+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:56.416 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:56.410+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:56.495 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:56.488+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:56.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:56.681+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:56.691 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:56.685+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:56.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:56.832+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:56.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:56.970+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:57.033 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:57.023+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:57.249 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:57.242+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:57.387 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:57.382+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:57.500 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:57.492+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:57.658 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:57.652+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:57.662 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:57.658+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:57.834 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:57.826+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:57.981 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:57.972+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:58.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:57.995+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:58.296 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:58.289+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:58.366 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:58.361+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:58.555 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:58.548+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:58.639 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:58.633+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:58.644 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:58.639+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:58.806 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:58.800+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:58.960 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:58.954+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:58.979 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:58.973+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:59.332 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:10:59.326+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:10:59.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:10:59.349+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:10:59.594 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:10:59.587+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:10:59.669 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:59.663+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:10:59.671 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:10:59.666+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:10:59.770 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:10:59.763+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:10:59.831 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:10:59.823+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e271 v271) 2019-11-29T17:10:59.983 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:10:59.977+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:10:59.995 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:59.989+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:10:59.995 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:10:59.990+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:00.320 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:00.314+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:00.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:00.386+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:00.644 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:00.629+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:00.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:00.633+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:00.646 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:00.634+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:00.731 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:00.718+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:00.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:00.967+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:01.005 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:00.997+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:01.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:01.001+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:01.318 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:01.312+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:01.412 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:01.406+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:01.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:01.683+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:01.692 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:01.685+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:01.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:01.686+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:01.722 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:01.716+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:01.979 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:01.973+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:02.061 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:02.053+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:02.061 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:02.056+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:02.346 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:02.332+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:02.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:02.424+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:02.648 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:02.639+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:02.649 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:02.644+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:02.697 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:02.691+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:02.771 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:02.765+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:02.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:02.974+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:03.073 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:03.068+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:03.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:03.070+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:03.329 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:03.322+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:03.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:03.393+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:03.694 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:03.688+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:03.695 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:03.690+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:03.698 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:03.694+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:03.751 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:03.745+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:03.976 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:03.970+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:04.069 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:04.063+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:04.072 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:04.068+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:04.349 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:04.343+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:04.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:04.410+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:04.657 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:04.651+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:04.747 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:04.741+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:04.752 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:04.747+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:04.785 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:04.778+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:04.852 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:11:04.844+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e271 v271) 2019-11-29T17:11:04.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:04.946+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:05.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:05.091+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:05.100 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:05.096+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:05.392 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:05.386+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:05.401 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:05.395+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:05.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:05.643+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:05.726 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:05.721+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:05.728 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:05.724+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:05.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:05.738+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:05.968 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:05.962+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:06.139 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:06.134+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:06.139 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:06.135+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:06.390 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:06.383+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:06.405 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:06.400+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:06.681 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:06.675+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:06.743 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:06.736+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:06.744 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:06.739+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:06.797 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:06.786+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:06.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:06.973+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:07.137 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:07.131+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:07.138 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:07.135+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:07.414 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:07.408+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:07.442 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:07.437+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:07.697 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:07.690+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:07.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:07.761+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:07.769 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:07.764+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:07.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:07.810+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:07.940 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:07.935+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:08.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:08.137+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:08.143 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:08.138+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:08.421 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:08.416+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:08.467 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:08.460+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:08.743 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:08.737+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:08.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:08.778+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:08.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:08.781+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:08.861 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:08.855+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:08.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:08.952+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:09.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:09.172+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:09.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:09.174+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:09.454 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:09.448+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:09.489 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:09.483+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:09.712 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:09.704+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:09.818 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:09.812+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:09.819 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:09.814+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:09.909 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:11:09.904+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e271 v271) 2019-11-29T17:11:09.916 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:09.911+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:09.927 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:09.922+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:10.192 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:10.187+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:10.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:10.191+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:10.459 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:10.454+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:10.511 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:10.505+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:10.673 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:10.668+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:10.841 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:10.832+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:10.841 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:10.835+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:10.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:10.911+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:10.936 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:10.930+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:11.241 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:11.236+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:11.244 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:11.240+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:11.432 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:11.426+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:11.565 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:11.559+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:11.721 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:11.715+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:11.861 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:11.855+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:11.862 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:11.857+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:11.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:11.905+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:11.960 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:11.954+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:12.283 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:12.276+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:12.284 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:12.279+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:12.446 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:12.440+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:12.605 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:12.598+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:12.759 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:12.752+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:12.873 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:12.863+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:12.873 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:12.867+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:12.915 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:12.909+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:12.916 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:12.910+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:13.329 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:13.324+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:13.331 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:13.327+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:13.412 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:13.406+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:13.609 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:13.601+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:13.759 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:13.753+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:13.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:13.906+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:13.915 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:13.909+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:13.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:13.913+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:13.969 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:13.960+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:14.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:14.302+0000 1694c700 -1 osd.3 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.964341+0000 front 2019-11-29T17:09:46.932791+0000 (oldest deadline 2019-11-29T17:10:32.148540+0000) 2019-11-29T17:11:14.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:14.303+0000 1694c700 -1 osd.3 271 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:14.445 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:11:14.439+0000 1694c700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.963403+0000 front 2019-11-29T17:09:46.929920+0000 (oldest deadline 2019-11-29T17:10:30.937217+0000) 2019-11-29T17:11:14.656 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:11:14.649+0000 1694c700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:44.555479+0000 front 2019-11-29T17:09:44.553003+0000 (oldest deadline 2019-11-29T17:10:27.384152+0000) 2019-11-29T17:11:14.755 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:11:14.743+0000 1694c700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.101522+0000 front 2019-11-29T17:09:46.111759+0000 (oldest deadline 2019-11-29T17:10:29.861251+0000) 2019-11-29T17:11:14.922 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:11:14.916+0000 1694c700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.783021+0000 front 2019-11-29T17:09:46.813557+0000 (oldest deadline 2019-11-29T17:10:30.004049+0000) 2019-11-29T17:11:14.927 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:14.919+0000 1694c700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:10:01.520171+0000 (oldest deadline 2019-11-29T17:10:41.520171+0000) 2019-11-29T17:11:14.928 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:11:14.923+0000 1694c700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e270) 2019-11-29T17:11:14.957 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:11:14.948+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e271 v271) 2019-11-29T17:11:14.967 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:11:14.961+0000 1694c700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:09:46.939847+0000 front 2019-11-29T17:09:46.814142+0000 (oldest deadline 2019-11-29T17:10:27.247257+0000) 2019-11-29T17:11:15.542 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:15.472+0000 1694c700 -1 osd.3 272 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:16.369 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:58:29.257 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.384 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:58:29.265 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.392 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:58:29.166 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.394 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:58:29.282 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.397 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:58:29.285 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.398 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:58:29.172 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.689 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:58:29.463 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.762 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:11:16.686+0000 1694c700 -1 osd.3 273 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102291 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414896208] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-11-29T17:11:16.842 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:58:29.730 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:11:16.850 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:58:29.738 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:10.775 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:12:10.537+0000 1694c700 -1 osd.0 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=10 ondisk+retry+write+known_if_redirected e274) 2019-11-29T17:12:11.592 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:12:11.580+0000 1694c700 -1 osd.0 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=10 ondisk+retry+write+known_if_redirected e274) 2019-11-29T17:12:13.178 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:12:13.055+0000 1694c700 -1 osd.0 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=10 ondisk+retry+write+known_if_redirected e274) 2019-11-29T17:12:45.461 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:59:58.350 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.498 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:59:58.386 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.518 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:59:58.406 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.545 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:59:58.318 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.550 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:59:58.323 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.566 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:59:58.450 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.638 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:05:59:58.526 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.663 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:05:59:58.551 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:12:45.702 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:05:59:58.476 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:02.721 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:02.714+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:03.608 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:03.602+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:03.724 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:03.718+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:04.409 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:04.401+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:04.580 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:04.574+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:04.740 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:04.731+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:04.757 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:04.751+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:05.282 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:05.275+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:05.380 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:05.373+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:05.624 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:05.618+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:05.786 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:05.780+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:05.792 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:05.786+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:05.998 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:05.992+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:06.264 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:06.257+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:06.419 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:06.413+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:06.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:06.658+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:06.767 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:06.760+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:06.823 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:06.815+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:07.053 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:07.045+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:07.274 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:07.268+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:07.421 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:07.413+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:07.468 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:07.462+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:07.697 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:07.690+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:07.792 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:07.786+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:07.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:07.840+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:08.022 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:08.017+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:08.245 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:08.237+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:08.395 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:08.387+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:08.457 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:08.450+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:08.709 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:08.702+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:08.825 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:08.818+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:08.846 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:08.838+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:09.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:09.004+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:09.212 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:09.206+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:09.445 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:09.438+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:09.466 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:09.459+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:09.687 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:09.680+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:09.805 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:09.798+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:09.848 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:09.841+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:10.063 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:10.057+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:10.220 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:10.214+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:10.438 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:10.431+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:10.494 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:10.486+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:10.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:10.673+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:10.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:10.772+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:10.866 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:10.858+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:11.049 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:11.043+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:11.219 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:11.212+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:11.426 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:11.419+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:11.466 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:11.459+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:11.694 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:11.687+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:11.743 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:11.737+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:11.853 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:11.845+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:12.029 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:12.023+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:12.217 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:12.211+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:12.448 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:12.441+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:12.466 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:12.460+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:12.656 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:12.650+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:12.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:12.771+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:12.890 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:12.881+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:13.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:13.034+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:13.177 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:13.170+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:13.467 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:13.460+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:13.488 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:13.480+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:13.691 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:13.684+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:13.833 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:13.826+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:13.935 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:13.929+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:14.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:14.007+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:14.164 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:14.157+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:14.476 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:14.470+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:14.491 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:14.485+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:14.699 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:14.693+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:14.860 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:14.853+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:14.888 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:14.883+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:15.037 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:15.025+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:15.166 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:15.160+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:15.455 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:15.448+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:15.483 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:15.476+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:15.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:15.731+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:15.862 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:15.855+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:15.867 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:15.861+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:16.089 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:16.082+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:16.180 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:16.173+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:16.439 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:16.434+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:16.458 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:16.448+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:16.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:16.749+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:16.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:16.896+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:16.906 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:16.900+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:17.093 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:17.087+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:17.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:17.152+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:17.408 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:17.402+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:17.493 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:17.485+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:17.768 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:17.762+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:17.862 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:17.856+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:17.875 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:17.865+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:18.141 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:18.134+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:18.185 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:18.179+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:18.411 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:18.404+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:18.499 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:18.493+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:18.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:18.781+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:18.877 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:18.871+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:18.879 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:18.873+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:19.169 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:19.162+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:19.181 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:19.175+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:19.387 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:19.381+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:19.537 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:19.530+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:19.797 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:19.792+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:19.905 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:19.900+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:19.928 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:19.921+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:20.141 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:20.132+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:20.218 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:20.211+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:20.389 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:20.381+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:20.511 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:20.503+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:20.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:20.834+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:20.897 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:20.889+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:20.948 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:20.942+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:21.185 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:21.178+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:21.199 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:21.190+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:21.391 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:21.384+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:21.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:21.498+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:21.876 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:21.871+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:21.898 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:21.891+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:21.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:21.972+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:22.159 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:22.151+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:22.208 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:22.190+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:22.368 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:22.363+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:22.496 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:22.486+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:22.869 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:22.863+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:22.917 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:22.908+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:22.994 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:22.988+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:23.143 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:23.137+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:23.223 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:23.216+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:23.418 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:23.411+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:23.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:23.502+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:23.879 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:23.870+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:23.929 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:23.921+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:24.032 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:24.024+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:24.164 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:24.153+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:24.253 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:24.246+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:24.413 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:24.407+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:24.520 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:24.511+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:24.836 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:24.829+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:24.976 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:24.971+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:25.066 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:25.060+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:25.141 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:25.132+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:25.300 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:25.293+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:25.455 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:25.450+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:25.550 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:25.542+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:25.892 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:25.884+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:25.965 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:25.960+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:26.032 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:26.026+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:26.180 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:26.173+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:26.328 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:26.321+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:26.420 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:26.414+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:26.578 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:26.570+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:26.907 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:26.899+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:26.945 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:26.939+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:27.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:27.013+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:27.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:27.139+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:27.339 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:27.331+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:27.410 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:27.400+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:27.605 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:27.597+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:27.936 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:27.928+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:27.957 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:27.953+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:28.016 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:28.012+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:28.145 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:28.137+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:28.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:28.380+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:28.434 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:28.429+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:28.575 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:28.567+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:28.969 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:28.961+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:28.982 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:28.975+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:29.065 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:29.058+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:29.192 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:29.184+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:29.413 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:29.405+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:29.427 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:29.420+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:29.548 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:29.541+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:29.984 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:29.977+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:30.018 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:30.011+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:30.100 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:30.093+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:30.148 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:30.141+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:30.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:30.382+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:30.425 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:30.419+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:30.547 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:30.541+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:30.963 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:30.957+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:31.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:31.003+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:31.114 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:31.107+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:31.205 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:31.195+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:31.400 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:31.389+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:31.440 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:31.434+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:31.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:31.499+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:32.011 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:32.001+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:32.011 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:32.004+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:32.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:32.088+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:32.255 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:32.249+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:32.396 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:32.390+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:32.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:32.461+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:32.529 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:32.524+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:33.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:33.018+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:33.041 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:33.035+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:33.100 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:33.094+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:33.293 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:33.286+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:33.404 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:33.398+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:33.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:33.486+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:33.503 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:33.496+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:34.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:34.003+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:34.061 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:34.054+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:34.081 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:34.075+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:34.331 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:34.324+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:34.418 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:34.413+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:34.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:34.444+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:34.534 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:34.527+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:35.022 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:35.012+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:35.023 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:35.011+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:35.075 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:35.053+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:35.303 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:35.296+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:35.478 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:35.472+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:35.484 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:35.474+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:35.538 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:35.531+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:35.565 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:13:35.560+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e277 v277) 2019-11-29T17:13:35.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:35.973+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:36.005 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:35.995+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:36.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:36.102+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:36.305 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:36.298+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:36.439 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:36.433+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:36.527 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:36.520+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:36.563 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:36.557+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:36.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:36.979+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:37.027 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:37.009+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:37.117 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:37.109+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:37.262 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:37.255+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:37.474 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:37.467+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:37.497 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:37.487+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:37.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:37.549+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:37.994 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:37.989+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:38.014 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:38.009+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:38.102 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:38.095+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:38.307 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:38.301+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:38.494 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:38.483+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:38.534 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:38.528+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:38.594 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:38.588+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:38.974 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:38.969+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:39.014 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:39.004+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:39.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:39.080+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:39.287 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:39.267+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:39.488 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:39.482+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:39.501 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:39.487+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:39.593 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:39.587+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:40.006 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:13:40.002+0000 1694c700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.258922+0000 front 2019-11-29T17:12:17.239120+0000 (oldest deadline 2019-11-29T17:13:02.982185+0000) 2019-11-29T17:13:40.014 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:40.009+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:40.145 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:13:40.132+0000 1694c700 -1 osd.6 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:17.459004+0000 front 2019-11-29T17:12:17.465612+0000 (oldest deadline 2019-11-29T17:13:01.853170+0000) 2019-11-29T17:13:40.307 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:13:40.286+0000 1694c700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.597943+0000 front 2019-11-29T17:12:20.478283+0000 (oldest deadline 2019-11-29T17:13:05.732805+0000) 2019-11-29T17:13:40.463 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:13:40.457+0000 1694c700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.243390+0000 front 2019-11-29T17:12:19.244947+0000 (oldest deadline 2019-11-29T17:13:03.683136+0000) 2019-11-29T17:13:40.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:13:40.512+0000 1694c700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:21.611951+0000 front 2019-11-29T17:12:21.616992+0000 (oldest deadline 2019-11-29T17:13:06.674775+0000) 2019-11-29T17:13:40.631 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:13:40.625+0000 1694c700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:19.145342+0000 front 2019-11-29T17:12:19.075631+0000 (oldest deadline 2019-11-29T17:13:04.249941+0000) 2019-11-29T17:13:40.730 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:13:40.719+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e277 v277) 2019-11-29T17:13:40.903 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:00:53.791 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:40.932 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:00:53.820 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:40.932 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:00:53.820 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:40.946 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:00:53.834 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:40.955 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:00:53.729 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:40.959 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:00:53.733 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:41.012 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:00:53.900 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:41.013 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:00:53.901 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:13:41.020 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:13:41.013+0000 1694c700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:12:20.231561+0000 front 2019-11-29T17:12:20.232424+0000 (oldest deadline 2019-11-29T17:13:03.713142+0000) 2019-11-29T17:13:41.033 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:00:53.805 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:14:37.848 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:37.841+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:38.286 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:14:38.249+0000 20e22700 -1 RGWWatcher::handle_error cookie 900109968 err (107) Transport endpoint is not connected 2019-11-29T17:14:38.899 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:38.890+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:39.726 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:39.719+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:39.879 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:39.871+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:39.950 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:39.941+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:40.148 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:40.141+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:40.718 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:40.710+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:40.844 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:40.836+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:41.002 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:40.994+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:41.060 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:41.054+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:41.193 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:41.185+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:41.673 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:41.665+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:41.729 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:41.719+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:41.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:41.878+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:42.037 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:42.032+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:42.053 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:42.046+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:42.220 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:42.213+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:42.274 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:42.267+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:42.652 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:42.638+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:42.752 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:42.746+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:42.926 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:42.918+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:43.072 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:43.065+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:43.086 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:43.080+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:43.233 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:43.226+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:43.280 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:43.273+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:43.682 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:43.675+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:43.739 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:43.732+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:43.883 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:43.875+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:44.031 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:44.024+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:44.106 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:44.101+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:44.247 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:44.242+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:44.249 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:44.243+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:44.717 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:44.707+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:44.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:44.771+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:44.854 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:44.846+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:45.022 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:45.014+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:45.143 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:45.136+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:45.240 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:45.233+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:45.292 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:45.286+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:45.730 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:45.724+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:45.809 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:45.803+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:45.870 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:45.864+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:46.036 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:46.030+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:46.104 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:46.097+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:46.254 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:46.248+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:46.255 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:46.250+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:46.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:46.758+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:46.796 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:46.791+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:46.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:46.896+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:47.069 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:47.062+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:47.122 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:47.117+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:47.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:47.118+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:47.250 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:47.244+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:47.250 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:47.244+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:47.776 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:47.763+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:47.821 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:47.815+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:47.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:47.948+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:48.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:48.058+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:48.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:48.113+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:48.124 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:48.118+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:48.209 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:48.203+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:48.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:48.205+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:48.751 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:48.745+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:48.870 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:48.865+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:48.957 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:48.951+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:49.092 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:49.086+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:49.145 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:49.139+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:49.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:49.143+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:49.180 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:49.174+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:49.247 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:49.241+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:49.730 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:49.724+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:49.873 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:49.867+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:49.960 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:49.954+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:50.073 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:50.067+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:50.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:50.140+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:50.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:50.142+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:50.225 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:50.219+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:50.241 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:50.235+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:50.752 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:50.745+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:50.886 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:50.881+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:50.921 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:50.916+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:51.103 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:51.097+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:51.134 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:51.127+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:51.140 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:51.136+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:51.225 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:51.219+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:51.285 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:51.279+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:51.808 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:51.795+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:51.863 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:51.857+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:51.907 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:51.900+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:51.992 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:14:51.974+0000 20e22700 -1 RGWWatcher::handle_error cookie 439981296 err (107) Transport endpoint is not connected 2019-11-29T17:14:52.109 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:52.103+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:52.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:52.115+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:52.121 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:52.117+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:52.223 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:52.217+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:52.287 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:52.283+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:52.766 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:52.758+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:52.848 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:52.842+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:52.960 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:52.953+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:53.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:53.082+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:53.089 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:53.084+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:53.148 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:53.141+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:53.262 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:53.255+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:53.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:53.293+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:53.818 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:53.809+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:53.854 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:53.849+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:53.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:53.925+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:54.106 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:54.099+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:54.107 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:54.102+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:54.173 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:54.166+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:54.294 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:54.288+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:54.318 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:54.314+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:54.852 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:54.846+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:54.904 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:54.898+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:54.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:54.955+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:55.143 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:55.137+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:55.144 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:55.139+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:55.151 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:55.145+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:55.321 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:55.315+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:55.364 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:55.358+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:55.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:55.866+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:55.958 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:55.951+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:55.978 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:55.973+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:56.116 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:56.110+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:56.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:56.118+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:56.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:56.119+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:56.282 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:56.276+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:56.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:56.311+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:56.836 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:56.830+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:56.993 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:56.986+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:56.993 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:56.989+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:14:57.016 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:57.009+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:57.140 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:57.135+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:57.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:57.159+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:57.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:57.161+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:57.277 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:57.271+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:57.346 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:57.341+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:57.879 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:57.873+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:57.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:57.944+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:57.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:57.946+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:14:57.983 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:57.976+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:58.129 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:58.123+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:58.205 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:58.199+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:58.206 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:58.201+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:58.269 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:58.262+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:58.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:58.386+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:58.915 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:58.908+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:58.987 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:58.982+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:58.988 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:58.983+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:14:58.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:14:58.988+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:14:59.167 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:14:59.160+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:14:59.224 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:14:59.217+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:14:59.260 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:59.253+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:14:59.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:14:59.256+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:14:59.355 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:14:59.348+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:14:59.884 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:14:59.878+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:14:59.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:59.960+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:14:59.969 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:14:59.964+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:00.038 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:00.031+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:00.156 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:00.151+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:00.223 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:00.217+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:00.224 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:00.219+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:00.255 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:00.249+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:00.395 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:00.389+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:00.900 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:00.891+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:00.985 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:00.980+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:00.986 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:00.983+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:01.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:01.005+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:01.178 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:01.168+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:01.220 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:01.215+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:01.221 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:01.217+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:01.302 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:01.297+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:01.411 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:01.406+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:01.873 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:01.866+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:01.983 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:01.977+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:02.024 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:02.018+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:02.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:02.020+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:02.194 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:02.188+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:02.270 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:02.264+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:02.274 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:02.269+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:02.306 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:02.300+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:02.397 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:02.392+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:02.877 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:02.871+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:03.022 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:03.016+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:03.032 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:03.025+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:03.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:03.027+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:03.250 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:03.244+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:03.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:03.259+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:03.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:03.263+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:03.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:03.345+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:03.418 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:03.412+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:03.861 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:03.855+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:04.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:03.999+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:04.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:04.004+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:04.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:04.008+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:04.237 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:04.230+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:04.244 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:04.238+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:04.245 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:04.240+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:04.314 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:04.307+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:04.385 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:04.380+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:04.848 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:04.841+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:04.964 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:04.958+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:05.044 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:05.038+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:05.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:05.042+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:05.196 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:05.190+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:05.225 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:05.217+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:05.227 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:05.221+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:05.338 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:05.332+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:05.378 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:05.371+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:05.857 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:05.850+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:05.979 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:05.972+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:06.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:06.040+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:06.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:06.043+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:06.194 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:06.186+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:06.268 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:06.262+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:06.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:06.265+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:06.310 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:06.303+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:06.335 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:06.330+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:06.874 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:06.867+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:06.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:06.987+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:07.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:07.037+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:07.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:07.041+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:07.214 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:07.207+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:07.268 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:07.262+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:07.271 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:07.266+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:07.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:07.292+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:07.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:07.323+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:07.875 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:07.868+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:08.019 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:08.013+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:08.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:08.015+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:08.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:08.019+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:08.213 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:08.207+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:08.321 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:08.314+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:08.321 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:08.317+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:08.328 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:08.323+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:08.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:08.346+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:08.923 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:08.917+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:08.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:08.968+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:08.976 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:08.971+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:09.040 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:09.033+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:09.244 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:09.237+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:09.314 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:09.308+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:09.336 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:09.329+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:09.366 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:09.359+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:09.367 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:09.362+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:09.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:09.912+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:10.015 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:10.010+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:10.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:10.011+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:10.053 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:10.046+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:10.224 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:10.219+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:10.300 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:10.295+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:10.366 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:10.359+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:10.370 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:10.363+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:10.372 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:10.366+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:10.907 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:10.901+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:10.998 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:10.992+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:10.999 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:10.994+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:11.050 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:11.044+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:11.093 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:15:11.086+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e282 v282) 2019-11-29T17:15:11.180 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:11.174+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:11.352 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:11.343+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:11.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:11.399+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:11.405 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:11.399+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:11.410 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:11.405+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:11.959 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:11.952+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:12.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:12.000+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:12.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:12.003+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:12.088 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:12.080+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:12.184 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:12.177+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:12.361 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:12.355+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:12.402 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:12.395+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:12.403 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:12.397+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:12.460 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:12.453+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:12.961 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:12.955+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:13.055 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:13.047+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:13.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:13.049+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:13.110 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:13.104+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:13.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:13.152+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:13.318 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:13.312+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:13.377 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:02:26.266 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.382 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:02:26.269 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.386 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:13.379+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:13.386 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:13.381+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:13.390 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:02:26.278 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.404 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:02:26.283 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.407 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:02:26.181 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.415 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:02:26.188 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.504 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:13.498+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:13.556 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:02:26.444 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.570 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:02:26.458 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.580 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:02:26.353 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:15:13.932 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:13.925+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:14.081 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:14.075+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:14.084 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:14.079+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:14.092 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:14.085+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:14.155 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:14.148+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:14.309 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:14.303+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:14.419 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:14.412+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:14.420 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:14.414+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:14.460 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:14.454+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:14.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:14.892+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:15.085 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:15.076+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:15.124 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:15.117+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:15.133 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:15.127+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:15.136 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:15.130+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:15.331 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:15.324+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:15.426 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:15:15.419+0000 1694c700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.309284+0000 front 2019-11-29T17:13:57.332165+0000 (oldest deadline 2019-11-29T17:14:42.073861+0000) 2019-11-29T17:15:15.426 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:15.419+0000 1694c700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:14:06.813069+0000 (oldest deadline 2019-11-29T17:14:46.813069+0000) 2019-11-29T17:15:15.428 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:15.421+0000 1694c700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e281) 2019-11-29T17:15:15.891 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:15:15.884+0000 1694c700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.332561+0000 front 2019-11-29T17:13:57.264153+0000 (oldest deadline 2019-11-29T17:14:38.861590+0000) 2019-11-29T17:15:16.059 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:15:16.052+0000 1694c700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:57.943331+0000 front 2019-11-29T17:13:52.447187+0000 (oldest deadline 2019-11-29T17:14:37.672437+0000) 2019-11-29T17:15:16.143 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:15:16.135+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e282 v282) 2019-11-29T17:15:16.165 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:16.158+0000 1694c700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:55.439157+0000 front 2019-11-29T17:13:55.283695+0000 (oldest deadline 2019-11-29T17:14:41.150716+0000) 2019-11-29T17:15:16.166 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:16.160+0000 1694c700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:16.179 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:15:16.170+0000 1694c700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:53.914625+0000 front 2019-11-29T17:13:54.050783+0000 (oldest deadline 2019-11-29T17:14:39.090213+0000) 2019-11-29T17:15:16.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:15:16.326+0000 1694c700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:13:56.543381+0000 front 2019-11-29T17:13:56.535623+0000 (oldest deadline 2019-11-29T17:14:39.429629+0000) 2019-11-29T17:15:16.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:15:16.904+0000 1694c700 -1 osd.0 283 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:102662 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e282) 2019-11-29T17:15:17.136 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:17.130+0000 1694c700 -1 osd.3 283 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:102684 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:15:18.202 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:15:18.171+0000 1694c700 -1 osd.3 284 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:102710 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439981296] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-29T17:17:13.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:13.408+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:14.377 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:14.365+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:15.029 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:15.018+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:15.343 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:15.335+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:15.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:15.986+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:16.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:16.047+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:16.078 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:16.070+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:16.306 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:16.300+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:16.653 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:16.646+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:16.930 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:16.924+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:16.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:16.967+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:17.032 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:17.027+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:17.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:17.032+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:17.275 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:17.269+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:17.316 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:17.307+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:17.701 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:17.694+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:17.945 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:17.937+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:17.990 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:17.983+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:18.002 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:17.997+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:18.050 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:18.044+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:18.265 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:18.259+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:18.352 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:18.344+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:18.669 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:18.662+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:18.913 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:18.908+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:18.998 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:18.992+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:19.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:18.998+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:19.079 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:19.072+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:19.306 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:19.299+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:19.315 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:19.308+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:19.671 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:19.664+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:19.920 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:19.913+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:19.997 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:19.990+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:20.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:20.008+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:20.041 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:20.034+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:20.295 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:20.289+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:20.352 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:20.347+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:20.725 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:20.717+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:20.941 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:20.923+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:20.997 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:20.990+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:21.051 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:21.044+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:21.060 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:21.053+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:21.336 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:21.329+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:21.336 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:21.330+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:21.700 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:21.692+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:21.965 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:21.958+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:22.032 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:22.026+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:22.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:22.035+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:22.064 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:22.057+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:22.302 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:22.295+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:22.335 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:22.330+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:22.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:22.730+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:22.990 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:22.985+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:23.023 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:23.017+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:23.070 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:23.063+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:23.086 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:23.079+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:23.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:23.287+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:23.341 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:23.334+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:23.766 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:23.759+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:23.967 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:23.961+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:23.996 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:23.989+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:24.120 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:24.110+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:24.129 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:24.120+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:24.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:24.298+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:24.374 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:24.367+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:24.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:24.765+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:24.921 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:24.915+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:25.008 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:24.999+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:25.098 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:25.092+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:25.101 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:25.094+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:25.340 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:25.331+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:25.430 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:25.418+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:25.785 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:25.777+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:25.912 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:25.906+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:25.999 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:25.986+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:26.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:26.055+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:26.077 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:26.071+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:26.303 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:26.298+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:26.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:26.409+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:26.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:26.826+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:26.965 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:26.958+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:27.016 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:27.010+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:27.068 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:27.063+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:27.080 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:27.073+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:27.326 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:27.322+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:27.456 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:27.449+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:27.830 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:27.821+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:27.957 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:27.947+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:27.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:27.990+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:28.043 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:28.037+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:28.113 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:28.106+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:28.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:28.346+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:28.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:28.411+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:28.821 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:28.815+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:28.915 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:28.908+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:29.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:29.011+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:29.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:29.038+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:29.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:29.140+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:29.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:29.377+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:29.474 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:29.467+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:29.837 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:29.830+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:29.969 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:29.958+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:30.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:30.037+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:30.056 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:30.047+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:30.203 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:30.196+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:30.356 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:30.350+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:30.458 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:30.452+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:30.810 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:30.803+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:31.018 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:31.000+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:31.027 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:31.020+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:31.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:31.079+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:31.223 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:31.217+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:31.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:31.375+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:31.471 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:31.461+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:31.783 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:31.775+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:31.964 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:31.956+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:32.067 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:32.060+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:32.070 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:32.063+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:32.194 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:32.187+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:32.414 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:32.407+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:32.504 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:32.497+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:32.744 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:32.737+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:32.971 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:32.964+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:33.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:33.023+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:33.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:33.032+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:33.238 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:33.232+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:33.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:33.378+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:33.488 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:33.481+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:33.703 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:33.697+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:33.966 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:33.959+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:34.051 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:34.044+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:34.057 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:34.049+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:34.280 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:34.273+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:34.436 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:34.428+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:34.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:34.513+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:34.726 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:34.720+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:34.957 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:34.949+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:35.074 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:35.068+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:35.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:35.087+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:35.330 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:35.319+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:35.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:35.408+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:35.529 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:35.523+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:35.754 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:35.748+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:35.930 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:35.922+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:36.059 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:36.053+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:36.090 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:36.084+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:36.311 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:36.304+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:36.466 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:36.460+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:36.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:36.550+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:36.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:36.791+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:36.913 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:36.904+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:37.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:37.035+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:37.052 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:37.044+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:37.342 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:37.335+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:37.489 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:37.481+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:37.609 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:37.602+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:37.827 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:37.819+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:37.955 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:37.948+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:38.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:37.998+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:38.087 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:38.081+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:38.355 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:38.346+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:38.512 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:38.506+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:38.653 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:38.647+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:38.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:38.792+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:38.979 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:38.971+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:39.049 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:39.043+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:39.067 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:39.060+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:39.344 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:39.337+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:39.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:39.501+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:39.707 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:39.700+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:39.846 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:39.840+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:39.956 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:39.948+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:40.069 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:40.062+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:40.098 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:40.090+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:40.396 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:40.390+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:40.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:40.553+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:40.755 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:40.748+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:40.810 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:40.803+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:40.959 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:40.953+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:41.094 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:41.087+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:41.116 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:41.110+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:41.376 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:41.366+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:41.518 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:41.510+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:41.726 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:41.720+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:41.785 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:41.779+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:41.925 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:41.917+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:42.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:42.043+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:42.167 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:42.155+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:42.357 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:42.345+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:42.499 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:42.493+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:42.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:42.699+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:42.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:42.818+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:42.954 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:42.948+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:43.021 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:43.015+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:43.125 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:43.118+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:43.321 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:43.314+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:43.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:43.476+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:43.751 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:43.744+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:43.800 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:43.794+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:44.009 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:44.002+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:44.068 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:44.061+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:44.102 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:44.096+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:44.330 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:44.324+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:44.441 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:44.435+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:44.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:44.760+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:44.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:44.815+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:45.046 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:45.039+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:45.078 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:45.071+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:45.109 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:45.096+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:45.294 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:45.286+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:45.487 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:45.480+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:45.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:17:45.810+0000 1694c700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:29.563897+0000 front 2019-11-29T17:16:29.244511+0000 (oldest deadline 2019-11-29T17:17:13.320898+0000) 2019-11-29T17:17:45.861 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:17:45.853+0000 1694c700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.170268+0000 front 2019-11-29T17:16:30.271452+0000 (oldest deadline 2019-11-29T17:17:15.905214+0000) 2019-11-29T17:17:46.075 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:17:46.067+0000 1694c700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.200543+0000 front 2019-11-29T17:16:30.641511+0000 (oldest deadline 2019-11-29T17:17:15.956910+0000) 2019-11-29T17:17:46.097 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:17:46.084+0000 1694c700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.197575+0000 front 2019-11-29T17:16:31.196230+0000 (oldest deadline 2019-11-29T17:17:16.454033+0000) 2019-11-29T17:17:46.134 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:17:46.126+0000 1694c700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:30.179025+0000 front 2019-11-29T17:16:30.002522+0000 (oldest deadline 2019-11-29T17:17:14.671698+0000) 2019-11-29T17:17:46.325 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:17:46.317+0000 1694c700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:31.002502+0000 front 2019-11-29T17:16:30.903118+0000 (oldest deadline 2019-11-29T17:17:15.693410+0000) 2019-11-29T17:17:46.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:17:46.476+0000 1694c700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:16:32.105393+0000 front 2019-11-29T17:16:31.983935+0000 (oldest deadline 2019-11-29T17:17:16.434257+0000) 2019-11-29T17:17:46.638 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:17:46.632+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e287 v287) 2019-11-29T17:18:19.010 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:18.932+0000 38279700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-29T17:18:19.066 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:19.057+0000 3b27f700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-29T17:18:19.233 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:19.226+0000 35273700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-29T17:18:20.443 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:20.436+0000 3a27d700 -1 osd.0 292 build_incremental_map_msg missing incremental map 292 2019-11-29T17:18:20.843 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:20.825+0000 3927b700 -1 osd.0 292 build_incremental_map_msg missing incremental map 292 2019-11-29T17:18:20.964 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:20.901+0000 3aa7e700 -1 osd.0 292 build_incremental_map_msg missing incremental map 292 2019-11-29T17:18:21.081 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:20.987+0000 38279700 -1 osd.0 292 build_incremental_map_msg missing incremental map 292 2019-11-29T17:18:21.204 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:21.085+0000 38279700 -1 osd.0 292 build_incremental_map_msg missing incremental map 292 2019-11-29T17:18:21.720 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:21.711+0000 124ad700 -1 osd.6 293 build_incremental_map_msg missing incremental map 291 2019-11-29T17:18:43.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:43.718+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:44.205 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:44.199+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:44.683 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:44.676+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:44.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:44.767+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:45.247 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:45.240+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:45.372 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:18:45.362+0000 20e22700 -1 RGWWatcher::handle_error cookie 1472904192 err (107) Transport endpoint is not connected 2019-11-29T17:18:45.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:45.704+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:45.741 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:45.735+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:46.226 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:46.219+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:46.692 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:46.686+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:46.764 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:46.758+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:47.239 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:47.233+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:47.440 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:47.433+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:47.730 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:47.723+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:47.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:47.784+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:48.206 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:48.199+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:48.486 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:48.480+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:48.749 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:48.742+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:48.837 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:48.830+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:49.128 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:49.120+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:49.257 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:49.252+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:49.441 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:49.432+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:49.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:49.474+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:49.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:49.740+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:49.809 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:49.803+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:50.095 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:50.085+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:50.169 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:50.165+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:50.299 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:50.293+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:50.456 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:50.450+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:50.484 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:50.477+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:50.742 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:50.735+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:50.791 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:50.785+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:51.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:51.108+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:51.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:51.186+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:51.321 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:51.312+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:51.462 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:51.456+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:51.501 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:51.494+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:51.718 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:51.711+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:51.829 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:51.821+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:52.130 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:52.125+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:52.207 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:52.203+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:52.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:52.346+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:52.443 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:52.437+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:52.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:52.475+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:52.755 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:52.749+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:52.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:52.784+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:53.114 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:53.108+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:53.257 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:53.241+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:53.334 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:53.327+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:53.456 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:53.447+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:53.529 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:53.522+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:53.758 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:53.752+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:53.817 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:53.811+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:54.139 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:54.132+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:54.220 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:54.215+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:54.343 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:54.336+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:54.501 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:54.495+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:54.582 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:54.575+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:54.741 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:54.733+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:54.791 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:54.785+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:55.167 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:55.161+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:55.216 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:55.212+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:55.371 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:55.365+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:55.535 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:55.529+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:55.580 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:55.574+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:55.767 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:55.760+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:55.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:55.834+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:56.180 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:56.173+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:56.183 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:56.179+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:56.385 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:56.379+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:56.521 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:56.516+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:56.601 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:56.594+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:56.791 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:56.785+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:56.827 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:56.821+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:57.186 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:57.181+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:18:57.189 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:57.184+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:57.206 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:57.200+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:57.386 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:57.380+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:57.538 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:57.532+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:57.628 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:57.620+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:57.821 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:57.814+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:57.826 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:57.820+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:58.239 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:58.233+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:18:58.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:58.237+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:58.263 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:58.256+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:58.350 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:58.345+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:58.530 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:58.523+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:58.617 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:58.610+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:58.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:58.784+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:58.865 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:58.858+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:18:59.236 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:18:59.230+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:18:59.249 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:59.242+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:18:59.252 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:18:59.247+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:18:59.397 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:18:59.390+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:18:59.546 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:18:59.539+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:18:59.622 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:18:59.615+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:18:59.833 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:18:59.825+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:18:59.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:18:59.848+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:00.196 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:00.189+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:00.274 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:00.268+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:00.276 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:00.271+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:00.383 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:00.377+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:00.500 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:00.494+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:00.611 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:00.603+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:00.680 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:19:00.674+0000 20e22700 -1 RGWWatcher::handle_error cookie 438410640 err (107) Transport endpoint is not connected 2019-11-29T17:19:00.814 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:00.807+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:00.823 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:00.815+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:01.227 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:01.220+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:01.238 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:01.230+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:01.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:01.233+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:01.339 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:01.331+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:01.469 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:01.463+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:01.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:01.599+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:01.811 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:01.804+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:01.871 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:01.864+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:02.223 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:02.215+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:02.245 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:02.238+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:02.246 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:02.239+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:02.351 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:02.344+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:02.484 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:02.475+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:02.646 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:02.640+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:02.769 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:02.763+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:02.886 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:02.870+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:03.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:03.190+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:03.282 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:03.275+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:03.283 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:03.277+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:03.350 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:03.342+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:03.486 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:03.479+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:03.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:03.618+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:03.728 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:03.721+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:03.875 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:03.867+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:04.219 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:04.211+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:04.331 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:04.321+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:04.333 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:04.327+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:04.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:04.344+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:04.510 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:04.503+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:04.627 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:04.619+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:04.703 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:04.696+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:04.705 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:04.700+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:04.921 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:04.913+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:05.200 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:05.192+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:05.335 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:05.327+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:05.338 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:05.331+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:05.341 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:05.336+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:05.526 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:05.519+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:05.631 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:05.609+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:05.678 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:05.672+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:05.680 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:05.675+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:05.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:05.911+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:06.233 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:06.226+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:06.319 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:06.313+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:06.321 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:06.316+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:06.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:06.323+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:06.551 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:06.544+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:06.602 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:06.594+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:06.653 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:06.645+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:06.653 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:06.647+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:06.905 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:06.898+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:07.209 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:07.201+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:07.307 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:07.299+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:07.307 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:07.301+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:07.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:07.361+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:07.530 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:07.523+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:07.610 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:07.602+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:07.699 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:07.692+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:07.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:07.696+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:07.887 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:07.879+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:08.204 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:08.198+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:08.277 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:08.270+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:08.280 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:08.274+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:08.362 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:08.355+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:08.562 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:08.556+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:08.648 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:08.640+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:08.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:08.719+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:08.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:08.721+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:08.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:08.862+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:09.236 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:09.230+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:09.239 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:09.230+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:09.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:09.234+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:09.328 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:09.320+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:09.521 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:09.514+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:09.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:09.642+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:09.722 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:09.715+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:09.725 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:09.719+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:09.856 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:09.849+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:10.249 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:10.243+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:10.252 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:10.244+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:10.253 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:10.247+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:10.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:10.316+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:10.519 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:10.513+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:10.633 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:10.626+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:10.709 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:10.700+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:10.709 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:10.704+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:10.895 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:10.888+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:11.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:11.200+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:11.210 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:11.204+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:11.260 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:11.253+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:11.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:11.278+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:11.521 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:11.514+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:11.687 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:11.680+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:11.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:11.727+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:11.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:11.730+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:11.869 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:11.862+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:12.198 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:12.192+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:12.200 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:12.194+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:12.255 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:12.249+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:12.287 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:12.280+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:12.512 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:12.500+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:12.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:12.664+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:12.714 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:12.707+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:12.714 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:12.709+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:12.838 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:12.831+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:13.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:13.152+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:13.159 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:13.153+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:13.247 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:13.240+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:13.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:13.299+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:13.543 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:13.536+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:13.681 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:13.675+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:13.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:13.726+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:13.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:13.727+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:13.869 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:13.861+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:14.129 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:14.121+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:14.130 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:14.124+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:14.233 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:14.224+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:14.338 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:14.328+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:14.444 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:06:27.332 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.455 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:06:27.343 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.461 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:06:27.349 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.464 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:06:27.351 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.476 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:06:27.250 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.485 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:06:27.259 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.510 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:14.503+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:14.690 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:06:27.577 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.704 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:14.697+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:14.704 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:14.699+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:14.717 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:14.710+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:14.796 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:06:27.682 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.852 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:06:27.626 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:19:14.903 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:14.898+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:15.137 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:15.129+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:15.138 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:15.132+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:15.220 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:15.214+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:15.359 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:15.351+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:15.477 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:15.471+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:15.688 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:15.682+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:15.720 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:15.713+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:15.720 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:15.715+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:15.918 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:15.912+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:16.118 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:16.111+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:16.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:16.115+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:16.226 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:16.220+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:16.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:16.346+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:16.521 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:16.514+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:16.733 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:16.724+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:16.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:16.729+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:16.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:16.731+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:16.893 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:16.884+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:16.895 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:19:16.886+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e293 v293) 2019-11-29T17:19:17.173 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:17.164+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:17.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:17.168+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:17.279 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:17.272+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:17.320 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:17.313+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:17.493 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:17.487+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:17.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:17.736+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:17.758 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:17.752+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:17.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:17.755+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:17.849 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:17.842+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:18.155 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:18.145+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:18.155 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:18.148+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:18.317 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:18.310+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:18.324 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:18.318+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:18.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:18.474+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:18.709 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:18.703+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:18.793 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:18.787+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:18.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:18.789+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:18.877 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:18.870+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:19.204 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:19.195+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:19.205 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:19.199+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:19.283 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:19.277+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:19.292 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:19.285+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:19.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:19.474+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:19.729 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:19.722+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:19.752 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:19.745+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:19.753 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:19.747+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:19.888 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:19.882+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:20.234 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:20.221+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:20.234 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:20.229+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:20.333 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:20.327+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:20.340 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:20.333+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:20.514 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:20.508+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:20.727 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:20.721+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:20.742 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:20.737+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:20.745 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:20.739+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:20.881 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:20.874+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:21.202 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:21.198+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:21.204 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:21.200+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:21.295 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:21.284+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:21.349 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:21.339+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:21.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:21.475+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:21.710 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:21.703+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:21.710 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:21.705+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:21.716 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:21.710+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:21.914 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:19:21.910+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e293 v293) 2019-11-29T17:19:21.924 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:21.918+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:22.224 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:22.217+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:22.225 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:22.220+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:22.267 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:22.248+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:22.389 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:22.382+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:22.437 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:22.431+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:22.771 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:22.745+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:22.771 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:22.739+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:22.772 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:22.742+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:22.957 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:22.951+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:23.255 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:23.249+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:23.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:23.255+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:23.291 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:23.285+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:23.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:23.386+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:23.399 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:23.393+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:23.751 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:23.744+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:23.751 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:23.746+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:23.755 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:23.748+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:24.015 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:24.007+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:24.275 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:24.266+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:24.276 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:24.269+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:24.346 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:24.339+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:24.375 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:24.369+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:24.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:24.378+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:24.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:24.720+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:24.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:24.722+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:24.745 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:24.739+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:25.063 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:25.056+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:25.308 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:25.300+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:25.309 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:25.304+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:25.356 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:25.350+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:25.412 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:25.406+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:25.420 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:25.413+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:25.749 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:25.742+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:25.753 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:25.746+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:25.754 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:25.749+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:26.020 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:26.012+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:26.275 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:26.271+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:26.279 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:26.273+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:26.361 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:26.355+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:26.408 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:26.402+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:26.458 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:26.451+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:26.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:26.712+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:26.721 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:26.716+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:26.757 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:26.745+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:26.927 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:19:26.920+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e293 v293) 2019-11-29T17:19:26.979 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:26.972+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:27.316 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:27.309+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:27.317 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:27.312+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:27.372 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:27.366+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:27.413 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:27.406+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:27.430 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:27.424+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:27.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:27.721+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:27.728 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:27.723+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:27.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:27.793+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:27.994 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:27.987+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:28.281 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:28.272+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:28.283 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:28.276+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:28.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:28.379+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:28.419 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:28.413+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:28.451 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:28.445+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:28.698 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:28.692+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:28.699 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:28.694+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:28.820 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:28.814+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:28.973 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:28.964+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:29.319 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:29.311+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:29.320 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:29.315+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:29.410 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:29.404+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:29.428 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:29.422+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:29.452 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:29.446+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:29.741 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:29.734+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:29.742 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:29.737+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:29.809 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:29.802+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:30.008 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:30.001+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:30.315 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:30.310+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:30.317 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:30.313+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:30.436 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:30.429+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:30.457 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:30.442+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:30.471 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:30.464+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:30.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:30.740+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:30.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:30.742+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:30.779 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:30.772+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:30.999 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:30.992+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:31.329 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:31.322+0000 1694c700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:18:16.444735+0000 (oldest deadline 2019-11-29T17:18:56.444735+0000) 2019-11-29T17:19:31.330 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:31.325+0000 1694c700 -1 osd.0 293 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:31.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:19:31.430+0000 1694c700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:01.358430+0000 front 2019-11-29T17:18:01.284379+0000 (oldest deadline 2019-11-29T17:18:47.102962+0000) 2019-11-29T17:19:31.452 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:19:31.445+0000 1694c700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:03.237602+0000 front 2019-11-29T17:18:03.282513+0000 (oldest deadline 2019-11-29T17:18:48.996663+0000) 2019-11-29T17:19:31.460 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:19:31.453+0000 1694c700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:00.402235+0000 front 2019-11-29T17:18:00.422508+0000 (oldest deadline 2019-11-29T17:18:43.807124+0000) 2019-11-29T17:19:31.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:31.694+0000 1694c700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:59.973770+0000 front 2019-11-29T17:18:00.848051+0000 (oldest deadline 2019-11-29T17:18:44.617126+0000) 2019-11-29T17:19:31.703 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:31.698+0000 1694c700 -1 osd.3 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:31.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:19:31.736+0000 1694c700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:18:02.741694+0000 front 2019-11-29T17:18:02.810306+0000 (oldest deadline 2019-11-29T17:18:48.613810+0000) 2019-11-29T17:19:31.986 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:19:31.969+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e293 v293) 2019-11-29T17:19:32.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:19:32.042+0000 1694c700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:17:58.423043+0000 front 2019-11-29T17:18:03.711714+0000 (oldest deadline 2019-11-29T17:18:43.679249+0000) 2019-11-29T17:19:32.470 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:19:32.387+0000 1694c700 -1 osd.0 294 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-29T17:19:32.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:32.724+0000 1694c700 -1 osd.3 294 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:19:33.793 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:19:33.779+0000 1694c700 -1 osd.3 295 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:103095 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 438410640] snapc 0=[] ondisk+write+known_if_redirected e293) 2019-11-29T17:20:26.824 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:20:26.533+0000 1694c700 -1 osd.0 298 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=51 ondisk+retry+read+known_if_redirected e296) 2019-11-29T17:21:20.137 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:20.127+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:20.788 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:20.781+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:21.099 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:21.092+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:21.756 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:21.749+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:22.014 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:22.006+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:22.126 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:22.119+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:22.479 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:22.472+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:22.808 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:22.801+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:22.980 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:22.972+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:23.133 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:23.125+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:23.435 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:23.427+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:23.479 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:23.470+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:23.512 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:23.504+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:23.819 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:23.811+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:23.947 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:23.940+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:24.107 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:24.101+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:24.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:24.450+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:24.480 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:24.473+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:24.562 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:24.554+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:24.777 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:24.771+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:24.981 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:24.974+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:25.080 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:25.073+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:25.156 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:25.148+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:25.477 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:25.469+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:25.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:25.518+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:25.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:25.527+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:25.761 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:25.754+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:26.005 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:25.998+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:26.114 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:26.107+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:26.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:26.130+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:26.491 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:26.484+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:26.494 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:26.485+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:26.494 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:26.485+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:26.765 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:26.758+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:27.045 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:27.038+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:27.083 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:27.076+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:27.138 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:27.132+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:27.480 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:27.472+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:27.500 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:27.493+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:27.522 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:27.514+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:27.750 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:27.743+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:28.087 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:28.081+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:28.134 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:28.127+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:28.180 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:28.170+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:28.457 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:28.450+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:28.514 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:28.506+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:28.524 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:28.516+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:28.803 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:28.797+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:29.129 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:29.118+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:29.140 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:29.132+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:29.182 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:29.173+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:29.490 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:29.484+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:29.507 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:29.500+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:29.541 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:29.534+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:29.837 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:29.829+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:30.112 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:30.104+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:30.116 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:30.110+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:30.163 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:30.156+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:30.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:30.475+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:30.501 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:30.494+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:30.545 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:30.533+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:30.878 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:30.872+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:31.080 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:31.073+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:31.106 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:31.099+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:31.167 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:31.159+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:31.469 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:31.462+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:31.494 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:31.487+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:31.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:31.518+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:31.893 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:31.885+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:32.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:32.036+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:32.143 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:32.135+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:32.145 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:32.137+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:32.456 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:32.450+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:32.479 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:32.470+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:32.562 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:32.554+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:32.943 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:32.933+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:33.116 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:33.109+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:33.156 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:33.149+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:33.180 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:33.169+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:33.485 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:33.479+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:33.489 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:33.481+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:33.589 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:33.581+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:33.956 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:33.949+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:34.105 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:34.097+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:34.177 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:34.169+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:34.212 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:34.204+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:34.520 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:34.513+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:34.527 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:34.520+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:34.575 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:34.569+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:34.938 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:34.931+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:35.141 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:35.133+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:35.170 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:35.163+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:35.225 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:35.217+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:35.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:35.483+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:35.504 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:35.499+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:35.618 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:35.610+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:35.983 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:35.977+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:36.106 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:36.091+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:36.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:36.182+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:36.212 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:36.206+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:36.478 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:36.472+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:36.538 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:36.528+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:36.608 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:36.600+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:36.980 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:36.971+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:37.091 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:37.083+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:37.232 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:37.221+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:37.253 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:37.244+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:37.433 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:37.426+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:37.584 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:37.575+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:37.620 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:37.609+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:37.942 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:37.935+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:38.091 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:38.083+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:38.241 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:38.234+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:38.284 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:38.279+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:38.458 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:38.453+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:38.544 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:38.536+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:38.578 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:38.570+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:38.990 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:38.983+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:39.061 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:39.053+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:39.291 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:39.283+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:39.320 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:39.314+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:39.455 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:39.449+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:39.502 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:39.494+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:39.585 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:39.576+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:39.989 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:39.983+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:40.117 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:40.099+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:40.297 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:40.285+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:40.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:40.311+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:40.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:40.421+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:40.473 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:40.465+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:40.549 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:40.541+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:41.020 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:41.014+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:41.116 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:41.108+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:41.291 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:41.285+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:41.296 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:41.288+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:41.451 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:41.445+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:41.530 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:41.521+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:41.536 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:41.528+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:42.049 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:42.042+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:42.120 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:42.112+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:42.278 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:42.271+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:42.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:42.276+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:42.430 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:42.424+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:42.574 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:42.566+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:42.576 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:42.568+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:43.078 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:43.071+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:43.097 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:43.091+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:43.299 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:43.292+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:43.316 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:43.310+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:43.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:43.383+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:43.533 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:43.527+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:43.566 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:43.560+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:44.110 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:44.103+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:44.141 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:44.136+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:44.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:44.275+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:44.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:44.301+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:44.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:44.422+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:44.495 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:44.488+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:44.567 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:44.560+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:45.084 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:45.077+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:45.162 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:45.153+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:45.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:45.327+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:45.352 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:45.346+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:45.395 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:45.388+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:45.541 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:45.526+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:45.619 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:45.612+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:46.058 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:46.050+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:46.120 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:46.114+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:46.334 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:46.323+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:46.345 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:46.330+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:46.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:46.370+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:46.571 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:46.563+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:46.674 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:46.668+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:47.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:47.010+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:47.141 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:47.135+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:47.294 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:47.286+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:47.344 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:47.336+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:47.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:47.376+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:47.572 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:47.565+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:47.640 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:47.633+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:47.986 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:47.980+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:48.099 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:48.093+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:48.336 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:48.329+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:48.352 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:48.346+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:48.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:48.369+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:48.550 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:48.543+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:48.635 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:48.628+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:48.949 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:48.943+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:49.070 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:49.065+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:49.363 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:49.357+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:49.390 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:49.382+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:49.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:49.395+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:49.601 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:49.592+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:49.628 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:49.620+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:49.955 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:49.948+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:50.031 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:50.025+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:50.354 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:50.349+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:50.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:50.386+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:50.436 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:50.428+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:50.592 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:50.586+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:50.662 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:50.656+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:50.982 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:50.974+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:50.992 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:50.985+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:51.355 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:51.350+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:51.366 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:51.361+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:51.400 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:51.392+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:51.637 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:51.630+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:51.651 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:51.643+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:51.943 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:51.936+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:51.957 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:51.952+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:52.336 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:52.327+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:52.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:52.331+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:52.348 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:52.342+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:52.351 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:21:52.344+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e298 v298) 2019-11-29T17:21:52.458 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:52.446+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:52.643 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:52.635+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:52.645 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:52.638+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:52.930 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:52.923+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:52.991 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:52.983+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:53.296 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:53.285+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:53.296 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:53.287+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:53.393 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:53.387+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:53.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:53.489+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:53.610 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:53.599+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:53.662 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:53.655+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:53.890 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:53.884+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:54.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:53.998+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:54.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:54.244+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:54.251 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:54.247+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:54.411 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:54.405+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:54.525 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:54.518+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:54.606 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:54.599+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:54.671 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:54.663+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:54.857 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:54.850+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:55.054 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:55.048+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:55.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:55.243+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:55.252 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:55.248+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:55.443 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:55.436+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:55.565 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:55.553+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:55.634 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:55.627+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:55.649 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:55.642+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:55.840 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:55.834+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:56.105 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:56.099+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:56.224 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:56.215+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:56.226 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:56.220+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:56.488 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:56.482+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:56.596 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:56.589+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:56.663 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:56.657+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:56.678 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:21:56.672+0000 1694c700 -1 osd.6 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.496086+0000 front 2019-11-29T17:20:37.332161+0000 (oldest deadline 2019-11-29T17:21:21.977794+0000) 2019-11-29T17:21:56.808 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:21:56.802+0000 1694c700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.397556+0000 front 2019-11-29T17:20:38.399052+0000 (oldest deadline 2019-11-29T17:21:20.421795+0000) 2019-11-29T17:21:57.091 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:21:57.084+0000 1694c700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:35.200232+0000 front 2019-11-29T17:20:35.202653+0000 (oldest deadline 2019-11-29T17:21:19.252000+0000) 2019-11-29T17:21:57.238 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:57.231+0000 1694c700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:36.782849+0000 front 2019-11-29T17:20:36.800310+0000 (oldest deadline 2019-11-29T17:21:21.979802+0000) 2019-11-29T17:21:57.238 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:57.233+0000 1694c700 -1 osd.2 298 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:57.420 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:21:57.409+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e298 v298) 2019-11-29T17:21:57.464 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:21:57.458+0000 1694c700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:37.779074+0000 front 2019-11-29T17:20:37.786092+0000 (oldest deadline 2019-11-29T17:21:22.819360+0000) 2019-11-29T17:21:57.644 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:21:57.638+0000 1694c700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.636766+0000 front 2019-11-29T17:20:38.725089+0000 (oldest deadline 2019-11-29T17:21:24.512192+0000) 2019-11-29T17:21:57.676 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:21:57.669+0000 1694c700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:20:38.978315+0000 front 2019-11-29T17:20:38.980120+0000 (oldest deadline 2019-11-29T17:21:22.662979+0000) 2019-11-29T17:21:58.221 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:58.204+0000 1694c700 -1 osd.2 299 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:21:59.247 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:21:59.241+0000 1694c700 -1 osd.2 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103550 2.21s0 2:87c9582a:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.610_%ef%84%91%e9%90%89%e1%a0%81%e2%98%83%e9%90%89%ef%84%91%e9%90%89%eb%8c%8d%ef%84%91%ef%84%92%eb%8c%8b%e3%9c%83%eb%8c%8b%e1%a0%83%e1%a0%81:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e298) 2019-11-29T17:22:02.543 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:09:15.431 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.568 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:09:15.456 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.579 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:09:15.466 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.600 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:09:15.477 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.625 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:09:15.398 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.636 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:09:15.410 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.736 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:09:15.624 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.744 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:09:15.632 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:02.854 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:09:15.627 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:22:55.068 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:22:55.055+0000 20e22700 -1 RGWWatcher::handle_error cookie 5417298048 err (107) Transport endpoint is not connected 2019-11-29T17:22:55.749 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:22:55.738+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:22:56.140 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:22:56.132+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:22:56.478 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:22:56.471+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:22:56.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:22:56.472+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:22:56.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:22:56.754+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:22:57.097 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:22:57.090+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:22:57.453 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:22:57.443+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:22:57.488 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:22:57.479+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:22:57.502 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:22:57.496+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:22:57.789 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:22:57.780+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:22:58.070 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:22:58.063+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:22:58.458 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:22:58.451+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:22:58.504 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:22:58.497+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:22:58.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:22:58.516+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:22:58.823 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:22:58.815+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:22:59.068 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:22:59.059+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:22:59.485 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:22:59.479+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:22:59.487 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:22:59.480+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:22:59.494 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:22:59.488+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:22:59.566 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:22:59.558+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:22:59.832 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:22:59.823+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:00.111 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:00.104+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:00.499 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:00.485+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:00.500 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:00.489+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:00.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:00.497+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:00.559 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:00.550+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:00.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:00.839+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:00.848 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:00.842+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:01.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:01.152+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:01.475 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:01.468+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:01.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:01.474+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:01.527 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:01.521+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:01.564 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:01.557+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:01.822 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:01.815+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:01.884 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:01.875+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:02.176 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:02.169+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:02.461 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:02.454+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:02.494 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:02.488+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:02.543 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:02.537+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:02.615 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:02.609+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:02.822 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:02.816+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:02.883 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:02.876+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:03.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:03.152+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:03.435 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:03.429+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:03.497 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:03.490+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:03.522 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:03.515+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:03.614 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:03.607+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:03.825 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:03.819+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:03.923 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:03.916+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:04.211 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:04.205+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:04.428 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:04.421+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:04.496 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:04.490+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:04.512 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:04.505+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:04.591 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:04.583+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:04.857 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:04.851+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:04.911 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:04.904+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:05.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:05.227+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:05.441 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:05.424+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:05.474 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:05.467+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:05.504 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:05.497+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:05.554 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:05.546+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:05.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:05.850+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:05.936 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:05.929+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:06.235 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:06.228+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:06.409 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:06.401+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:06.481 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:06.475+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:06.539 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:06.532+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:06.540 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:06.531+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:06.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:06.847+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:06.857 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:06.851+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:06.949 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:06.943+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:07.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:07.227+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:07.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:07.421+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:07.497 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:07.488+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:07.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:07.512+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:07.575 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:07.562+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:07.911 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:07.903+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:07.912 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:07.906+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:07.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:07.978+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:08.226 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:08.218+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:08.470 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:08.463+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:08.482 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:08.475+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:08.515 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:08.508+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:08.591 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:08.584+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:08.933 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:08.928+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:08.934 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:08.929+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:09.031 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:09.024+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:09.250 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:09.243+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:09.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:09.426+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:09.462 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:09.448+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:09.521 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:09.515+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:09.638 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:09.630+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:09.989 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:09.980+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:09.993 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:09.984+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:10.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:10.007+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:10.299 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:10.288+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:10.426 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:10.419+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:10.504 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:10.496+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:10.559 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:10.553+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:10.628 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:10.622+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:10.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:10.948+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:10.956 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:10.951+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:10.969 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:10.963+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:11.325 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:11.318+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:11.406 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:11.400+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:11.476 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:11.469+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:11.530 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:11.523+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:11.643 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:11.636+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:11.914 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:11.908+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:11.917 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:11.912+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:11.989 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:11.983+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:12.293 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:12.286+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:12.422 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:12.416+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:12.444 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:12.437+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:12.584 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:12.577+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:12.627 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:12.620+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:12.875 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:12.867+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:12.876 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:12.870+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:12.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:12.967+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:13.345 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:13.338+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:13.444 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:13.437+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:13.473 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:13.467+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:13.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:13.594+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:13.659 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:13.652+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:13.874 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:13.867+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:13.876 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:13.871+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:14.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:14.004+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:14.392 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:14.385+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:14.437 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:14.431+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:14.449 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:14.441+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:14.654 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:14.646+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:14.730 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:14.717+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:14.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:14.902+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:14.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:14.904+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:14.966 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:14.960+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:15.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:15.410+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:15.422 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:15.411+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:15.466 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:15.460+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:15.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:15.694+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:15.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:15.699+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:15.878 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:15.872+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:15.882 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:15.877+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:15.934 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:15.927+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:16.399 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:16.392+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:16.402 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:16.397+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:16.488 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:16.482+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:16.671 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:16.663+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:16.674 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:16.668+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:16.889 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:16.883+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:16.891 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:16.883+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:16.893 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:16.887+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:17.396 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:17.391+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:17.412 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:17.404+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:17.534 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:17.524+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:17.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:17.618+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:17.717 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:17.709+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:17.918 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:17.913+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:17.927 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:10:30.814 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:17.932 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:10:30.820 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:17.934 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:17.928+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:17.935 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:17.930+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:17.951 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:10:30.725 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:17.958 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:10:30.732 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:17.961 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:10:30.848 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:17.966 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:10:30.853 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:18.048 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:10:30.936 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:18.061 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:10:30.835 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:18.065 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:10:30.953 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:23:18.409 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:18.403+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:18.456 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:18.451+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:18.543 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:18.538+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:18.674 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:18.666+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:18.754 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:18.746+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:18.945 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:18.939+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:18.946 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:18.940+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:18.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:18.950+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:19.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:19.392+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:19.474 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:19.467+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:19.547 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:19.540+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:19.694 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:19.687+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:19.713 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:19.707+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:19.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:19.924+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:19.944 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:19.929+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:19.945 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:19.933+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:20.371 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:20.364+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:20.470 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:20.463+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:20.589 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:20.578+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:20.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:20.740+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:20.751 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:20.743+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:20.898 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:20.891+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:20.904 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:20.897+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:20.937 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:20.929+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:21.377 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:21.367+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:21.488 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:21.480+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:21.544 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:21.537+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:21.730 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:21.723+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:21.762 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:21.755+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:21.945 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:21.932+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:21.959 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:21.937+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:21.961 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:21.954+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:22.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:22.370+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:22.529 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:22.523+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:22.575 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:22.568+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:22.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:22.736+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:22.761 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:22.754+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:22.975 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:22.969+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:22.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:22.971+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:22.981 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:22.975+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:23.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:23.372+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:23.524 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:23.517+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:23.533 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:23.527+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:23.747 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:23.740+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:23.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:23.761+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:23.824 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:23:23.443+0000 20e22700 -1 RGWWatcher::handle_error cookie 5468256896 err (107) Transport endpoint is not connected 2019-11-29T17:23:23.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:23.977+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:24.002 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:23.994+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:24.004 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:23.997+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:24.357 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:24.351+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:24.528 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:24.521+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:24.541 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:24.531+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:24.726 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:24.719+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:24.799 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:24.793+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:24.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:24.949+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:24.972 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:24.965+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:24.974 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:24.968+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:25.340 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:25.334+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:25.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:25.515+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:25.554 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:25.546+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:25.769 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:25.761+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:25.821 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:25.813+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:25.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:25.967+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:26.013 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:26.007+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:26.017 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:26.008+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:26.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:26.333+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:26.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:26.508+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:26.574 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:26.566+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:26.771 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:26.763+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:26.859 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:26.850+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:26.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:26.926+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:26.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:26.992+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:27.002 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:26.995+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:27.363 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:27.356+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:27.498 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:27.490+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:27.628 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:27.620+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:27.728 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:23:27.723+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e303 v303) 2019-11-29T17:23:27.775 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:27.766+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:27.904 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:27.896+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:27.964 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:27.958+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:27.965 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:27.960+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:28.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:28.003+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:28.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:28.005+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:28.408 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:28.401+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:28.466 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:28.458+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:28.586 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:28.577+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:28.798 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:28.790+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:28.868 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:28.860+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:28.978 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:28.972+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:28.979 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:28.974+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:29.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:29.037+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:29.050 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:29.041+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:29.420 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:29.413+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:29.451 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:29.440+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:29.631 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:29.623+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:29.792 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:29.784+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:29.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:29.890+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:30.031 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:30.024+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:30.032 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:30.027+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:30.046 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:30.037+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:30.049 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:30.041+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:30.426 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:30.418+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:30.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:30.425+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:30.623 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:30.614+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:30.792 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:30.784+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:30.857 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:30.850+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:31.060 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:31.052+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:31.062 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:31.057+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:31.071 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:31.064+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:31.072 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:31.066+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:31.423 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:31.415+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:31.432 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:31.426+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:31.578 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:31.569+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:31.845 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:31.835+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:31.882 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:31.873+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:32.027 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:32.020+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:32.027 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:32.022+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:32.077 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:32.064+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:32.077 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:32.068+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:32.417 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:32.406+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:32.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:32.475+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:32.624 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:32.616+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:32.749 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:23:32.741+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e303 v303) 2019-11-29T17:23:32.810 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:32.802+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:32.907 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:32.898+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:33.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:33.010+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:33.022 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:33.011+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:33.047 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:33.029+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:33.047 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:33.031+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:33.458 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:33.450+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:33.506 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:33.500+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:33.611 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:33.602+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:33.798 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:33.790+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:33.947 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:33.940+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:34.013 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:33.995+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:34.013 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:33.998+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:34.019 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:34.012+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:34.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:34.014+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:34.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:34.500+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:34.548 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:34.541+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:34.641 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:34.630+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:34.823 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:34.814+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:34.975 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:34.968+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:35.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:35.001+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:35.010 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:35.005+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:35.019 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:35.011+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:35.020 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:35.014+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:35.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:35.510+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:35.547 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:35.539+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:35.625 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:35.619+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:35.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:35.793+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:36.008 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:35.994+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:36.043 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:36.033+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:36.044 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:36.037+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:36.069 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:36.060+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:36.071 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:36.064+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:36.508 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:36.502+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:36.559 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:36.551+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:36.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:36.649+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:36.829 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:36.819+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:37.007 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:23:37.000+0000 1694c700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:12.976197+0000 front 2019-11-29T17:22:13.098067+0000 (oldest deadline 2019-11-29T17:22:56.445243+0000) 2019-11-29T17:23:37.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:37.011+0000 1694c700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.132710+0000 front 2019-11-29T17:22:13.133604+0000 (oldest deadline 2019-11-29T17:22:55.023117+0000) 2019-11-29T17:23:37.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:37.012+0000 1694c700 -1 osd.3 303 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:37.028 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:37.021+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:37.029 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:37.023+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:37.535 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:23:37.526+0000 1694c700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:10.223016+0000 front 2019-11-29T17:22:10.232985+0000 (oldest deadline 2019-11-29T17:22:56.069140+0000) 2019-11-29T17:23:37.616 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:23:37.610+0000 1694c700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:14.324427+0000 front 2019-11-29T17:22:14.314716+0000 (oldest deadline 2019-11-29T17:22:59.500656+0000) 2019-11-29T17:23:37.690 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:23:37.683+0000 1694c700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:13.118391+0000 front 2019-11-29T17:22:10.196078+0000 (oldest deadline 2019-11-29T17:22:56.055461+0000) 2019-11-29T17:23:37.812 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:23:37.800+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e303 v303) 2019-11-29T17:23:37.855 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:23:37.849+0000 1694c700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:22:11.638459+0000 front 2019-11-29T17:22:13.133233+0000 (oldest deadline 2019-11-29T17:22:56.131593+0000) 2019-11-29T17:23:38.007 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:37.998+0000 1694c700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:22:25.916725+0000 (oldest deadline 2019-11-29T17:23:05.916725+0000) 2019-11-29T17:23:38.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:23:38.002+0000 1694c700 -1 osd.0 303 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-11-29T17:23:38.363 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:38.237+0000 1694c700 -1 osd.3 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:23:39.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:23:39.254+0000 1694c700 -1 osd.3 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:103729 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5468256896] snapc 0=[] ondisk+write+known_if_redirected e303) 2019-11-29T17:24:38.255 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:24:38.210+0000 1694c700 -1 osd.0 308 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=21 ondisk+retry+write+known_if_redirected e306) 2019-11-29T17:25:07.998 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:07.991+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:09.022 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:09.016+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:09.992 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:09.986+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:10.949 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:10.943+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:11.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:11.968+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:12.942 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:12.936+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:13.940 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:13.934+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:14.959 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:14.953+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:15.977 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:15.970+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:16.970 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:16.964+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:17.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:17.978+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:18.967 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:18.961+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:19.949 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:19.943+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:20.954 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:20.947+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:21.938 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:21.932+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:22.979 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:22.974+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:23.939 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:23.934+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:24.980 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:24.974+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:26.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:26.001+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:27.043 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:27.038+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:28.091 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:28.086+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:29.059 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:29.054+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:29.534 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:29.527+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:29.799 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:29.793+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:30.025 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:30.020+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:30.538 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:30.530+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:30.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:30.757+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:31.003 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:30.998+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:31.222 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:31.213+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:31.536 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:31.529+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:31.785 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:31.779+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:31.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:31.957+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:31.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:31.960+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:32.237 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:32.230+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:32.570 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:32.563+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:32.743 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:32.736+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:32.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:32.958+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:32.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:32.961+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:33.005 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:32.997+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:33.196 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:33.189+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:33.611 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:33.604+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:33.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:33.755+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:33.936 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:33.929+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:33.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:33.931+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:33.991 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:33.984+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:34.252 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:34.245+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:34.270 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:34.263+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:34.495 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:34.488+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:34.582 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:34.574+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:34.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:34.717+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:34.951 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:34.943+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:34.968 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:34.960+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:34.970 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:34.964+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:35.227 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:35.220+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:35.266 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:35.258+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:35.529 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:35.523+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:35.603 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:35.595+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:35.766 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:35.759+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:35.991 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:35.983+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:35.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:35.965+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:35.992 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:35.967+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:36.225 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:36.216+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:36.274 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:36.267+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:36.572 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:36.564+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:36.656 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:36.650+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:36.734 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:36.727+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:36.954 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:36.946+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:36.960 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:36.954+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:36.960 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:36.955+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:37.257 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:37.251+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:37.292 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:37.284+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:37.541 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:37.535+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:37.631 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:37.623+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:37.787 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:37.780+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:37.915 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:37.909+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:37.919 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:37.914+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:37.943 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:37.936+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:38.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:38.216+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:38.302 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:38.295+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:38.580 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:38.573+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:38.666 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:38.659+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:38.742 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:38.735+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:38.903 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:38.897+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:38.904 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:38.899+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:38.908 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:38.900+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:39.213 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:39.206+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:39.315 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:39.307+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:39.562 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:39.555+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:39.639 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:39.632+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:39.756 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:39.749+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:39.872 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:39.860+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:39.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:39.888+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:39.895 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:39.890+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:40.173 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:40.165+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:40.282 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:40.274+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:40.607 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:40.599+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:40.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:40.643+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:40.800 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:40.792+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:40.874 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:40.867+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:40.917 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:40.912+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:40.919 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:40.915+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:41.162 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:41.156+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:41.253 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:41.246+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:41.573 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:41.566+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:41.651 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:41.644+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:41.844 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:41.836+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:41.859 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:41.851+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:41.885 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:41.878+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:41.886 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:41.881+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:42.174 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:42.168+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:42.297 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:42.289+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:42.534 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:42.528+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:42.657 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:42.650+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:42.878 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:42.868+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:42.906 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:42.899+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:42.928 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:42.921+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:42.928 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:42.923+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:43.227 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:43.220+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:43.350 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:43.342+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:43.572 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:43.564+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:43.648 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:43.641+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:43.862 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:43.855+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:43.886 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:43.880+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:43.888 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:43.883+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:43.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:43.955+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:44.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:44.243+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:44.343 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:44.337+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:44.578 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:44.572+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:44.618 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:44.613+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:44.832 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:44.824+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:44.900 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:44.895+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:44.901 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:44.896+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:44.937 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:44.931+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:45.296 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:45.290+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:45.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:45.387+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:45.579 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:45.571+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:45.667 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:45.660+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:45.824 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:45.813+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:45.881 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:45.875+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:45.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:45.879+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:45.977 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:45.970+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:46.272 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:46.265+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:46.359 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:46.350+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:46.560 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:46.554+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:46.657 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:46.649+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:46.826 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:46.818+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:46.863 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:46.857+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:46.865 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:46.861+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:46.942 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:46.936+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:47.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:47.260+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:47.356 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:47.350+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:47.573 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:47.564+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:47.621 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:47.613+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:47.837 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:47.831+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:47.839 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:47.834+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:47.880 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:47.873+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:47.912 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:47.904+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:48.256 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:48.251+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:48.367 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:48.359+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:48.579 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:48.572+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:48.585 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:48.579+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:48.829 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:48.824+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:48.831 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:48.826+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:48.867 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:48.859+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:48.963 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:48.954+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:49.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:49.261+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:49.394 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:49.387+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:49.544 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:49.536+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:49.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:49.539+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:49.865 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:49.858+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:49.866 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:49.860+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:49.883 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:49.876+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:49.961 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:49.950+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:50.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:50.378+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:50.402 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:50.395+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:50.503 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:50.496+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:50.594 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:50.586+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:50.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:50.876+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:50.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:50.878+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:50.910 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:50.903+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:50.988 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:50.981+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:51.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:51.393+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:51.403 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:51.396+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:51.518 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:51.500+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:51.629 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:51.622+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:51.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:51.894+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:51.903 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:51.897+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:51.929 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:51.921+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:51.965 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:51.957+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:52.409 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:52.402+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:52.448 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:52.440+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:52.523 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:52.516+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:52.618 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:52.603+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:52.934 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:52.927+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:52.936 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:52.929+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:52.960 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:52.953+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:52.977 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:52.970+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:53.450 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:53.443+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:53.498 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:53.492+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:53.566 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:53.559+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:53.570 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:53.564+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:53.950 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:53.943+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:53.952 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:53.947+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:53.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:53.963+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:53.991 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:53.969+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:54.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:54.493+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:54.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:54.498+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:54.549 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:54.540+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:54.618 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:54.611+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:54.965 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:54.957+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:54.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:54.960+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:54.967 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:54.959+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:54.995 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:54.988+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:55.468 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:55.460+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:55.547 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:55.538+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:55.549 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:55.541+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:55.597 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:55.589+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:55.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:55.952+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:55.995 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:55.987+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:55.999 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:55.993+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:56.037 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:56.028+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:56.462 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:56.456+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:56.527 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:56.518+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:56.599 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:56.590+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:56.601 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:56.594+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:56.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:56.976+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:56.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:56.978+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:56.987 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:56.979+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:57.027 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:57.019+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:57.505 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:57.496+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:57.556 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:57.549+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:57.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:57.550+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:57.597 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:57.586+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:57.970 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:57.963+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:58.016 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:58.008+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:58.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:58.012+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:58.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:58.063+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:58.487 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:58.481+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:58.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:58.547+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:58.586 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:58.578+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:58.607 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:58.600+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:58.964 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:58.956+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:59.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:59.011+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:59.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:59.013+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:25:59.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:25:59.091+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:25:59.453 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:25:59.441+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:25:59.576 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:25:59.568+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:25:59.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:25:59.593+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:25:59.605 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:25:59.597+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:25:59.941 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:25:59.933+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:25:59.999 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:59.991+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:25:59.999 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:25:59.993+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:00.152 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:26:00.145+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:26:00.469 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:26:00.461+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:26:00.559 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:26:00.552+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:26:00.573 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:00.565+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:26:00.596 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:26:00.590+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:26:00.915 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:26:00.907+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:26:00.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:00.950+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:26:00.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:00.951+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:01.160 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:26:01.152+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:26:01.486 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:26:01.478+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:26:01.525 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:26:01.517+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:26:01.603 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:01.595+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:26:01.613 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:26:01.607+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:26:01.898 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:26:01.890+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:26:01.955 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:01.946+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:26:01.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:01.949+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:02.146 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:26:02.137+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:26:02.468 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:26:02.459+0000 1694c700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:46.805043+0000 front 2019-11-29T17:24:47.025442+0000 (oldest deadline 2019-11-29T17:25:30.278079+0000) 2019-11-29T17:26:02.550 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:26:02.544+0000 1694c700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:48.915958+0000 front 2019-11-29T17:24:48.908757+0000 (oldest deadline 2019-11-29T17:25:33.567897+0000) 2019-11-29T17:26:02.582 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:02.574+0000 1694c700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:45.020803+0000 front 2019-11-29T17:24:50.469922+0000 (oldest deadline 2019-11-29T17:25:29.439964+0000) 2019-11-29T17:26:02.608 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:26:02.602+0000 1694c700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.493107+0000 front 2019-11-29T17:24:49.346614+0000 (oldest deadline 2019-11-29T17:25:34.012105+0000) 2019-11-29T17:26:02.860 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:26:02.851+0000 1694c700 -1 osd.6 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:44.232517+0000 front 2019-11-29T17:24:44.090313+0000 (oldest deadline 2019-11-29T17:25:32.838013+0000) 2019-11-29T17:26:02.933 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:02.926+0000 1694c700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:52.311313+0000 front 2019-11-29T17:24:50.118727+0000 (oldest deadline 2019-11-29T17:25:31.398233+0000) 2019-11-29T17:26:02.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:02.931+0000 1694c700 -1 osd.2 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:03.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:26:03.155+0000 1694c700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:24:49.492576+0000 front 2019-11-29T17:24:49.334479+0000 (oldest deadline 2019-11-29T17:25:29.717226+0000) 2019-11-29T17:26:03.251 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:26:03.233+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e308 v308) 2019-11-29T17:26:04.098 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:03.976+0000 1694c700 -1 osd.2 309 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:05.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:04.981+0000 1694c700 -1 osd.2 310 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:103950 2.1s0 2:8371f89d:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%85%e5%98%86%e8%94%8b%e9%90%8c%ed%88%8e%ef%84%91%eb%8c%8c%e9%90%8b%e3%9c%86%e3%9c%86%e9%90%8b%e9%90%8b%ef%84%92%e1%a0%81%e1%a0%83:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e308) 2019-11-29T17:26:18.690 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:13:31.578 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.698 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:13:31.586 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.736 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:13:31.623 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.737 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:13:31.625 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.808 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:13:31.582 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.820 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:13:31.589 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.912 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:13:31.800 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:18.957 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:13:31.844 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:19.111 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:13:31.884 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:26:41.345 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:41.267+0000 3aa7e700 -1 osd.2 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.418 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:41.341+0000 33a70700 -1 osd.2 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:41.410+0000 35a74700 -1 osd.5 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.430 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:41.420+0000 36275700 -1 osd.5 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:41.343+0000 39a7c700 -1 osd.2 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.452 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:26:41.385+0000 34271700 -1 osd.2 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:41.520+0000 3927b700 -1 osd.5 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:26:41.764 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:26:41.753+0000 38a7a700 -1 osd.5 312 build_incremental_map_msg missing incremental map 312 2019-11-29T17:27:04.856 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:27:04.832+0000 20e22700 -1 RGWWatcher::handle_error cookie 820327232 err (107) Transport endpoint is not connected 2019-11-29T17:27:11.962 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:11.957+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:13.012 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:13.007+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:14.047 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:14.042+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:15.039 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:15.033+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:16.027 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:16.022+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:17.048 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:17.039+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:18.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:18.078+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:18.592 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:14:31.480 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.599 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:14:31.487 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.608 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:14:31.496 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.615 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:14:31.501 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.626 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:14:31.399 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.637 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:14:31.410 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.691 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:14:31.578 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.714 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:14:31.602 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:18.776 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:14:31.549 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:27:19.095 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:19.090+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:19.120 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:27:18.921+0000 20e22700 -1 RGWWatcher::handle_error cookie 5440973200 err (107) Transport endpoint is not connected 2019-11-29T17:27:20.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:20.118+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:21.096 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:21.089+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:22.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:22.112+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:23.127 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:23.122+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:24.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:24.077+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:24.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:24.095+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:25.108 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:25.103+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:25.466 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:25.104+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:26.086 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:26.079+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:26.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:26.080+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:27.053 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:27.047+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:27.120 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:27.114+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:28.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:28.003+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:28.120 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:28.113+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:28.990 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:28.983+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:29.093 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:29.087+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:30.000 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:29.993+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:30.116 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:30.107+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:31.037 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:31.031+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:31.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:31.139+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:32.022 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:32.015+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:32.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:32.189+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:33.054 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:33.047+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:33.160 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:33.155+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:34.096 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:34.090+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:34.139 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:34.134+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:35.065 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:35.054+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:35.112 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:35.106+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:36.029 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:36.022+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:36.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:36.138+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:37.026 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:37.017+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:37.113 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:37.107+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:38.024 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:38.018+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:38.164 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:38.157+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:39.051 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:39.044+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:39.141 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:39.136+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:40.055 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:40.049+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:40.164 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:40.159+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:41.045 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:41.040+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:41.189 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:41.184+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:42.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:42.066+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:42.210 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:42.204+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:43.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:43.078+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:43.180 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:43.176+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:44.044 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:44.038+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:44.192 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:44.188+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:45.070 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:45.064+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:45.232 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:45.227+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:46.050 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:46.043+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:46.078 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:46.072+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:46.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:46.078+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:46.257 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:46.251+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:46.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:46.327+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:47.102 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:47.095+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:47.111 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:47.105+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:47.114 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:47.108+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:47.144 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:47.136+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:47.200 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:47.190+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:47.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:47.261+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:47.385 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:47.379+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:48.071 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:48.063+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:48.072 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:48.066+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:48.107 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:48.098+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:48.154 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:48.143+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:48.227 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:48.213+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:48.311 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:48.306+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:48.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:48.386+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:49.034 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:49.028+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:49.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:49.033+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:49.093 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:49.087+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:49.116 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:49.110+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:49.243 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:49.234+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:49.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:49.260+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:49.409 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:49.402+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:50.009 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:50.002+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:50.012 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:50.004+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:50.087 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:50.078+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:50.110 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:50.103+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:50.280 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:50.270+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:50.296 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:50.291+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:50.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:50.292+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:50.440 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:50.434+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:50.976 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:50.970+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:50.977 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:50.972+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:50.978 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:50.969+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:51.042 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:51.034+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:51.159 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:51.152+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:51.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:51.279+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:51.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:51.282+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:51.300 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:51.292+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:51.401 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:51.395+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:52.017 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:52.011+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:52.017 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:52.009+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:52.019 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:52.015+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:52.058 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:52.051+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:52.201 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:52.194+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:52.296 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:52.288+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:52.333 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:52.326+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:52.334 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:52.328+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:52.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:52.376+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:52.976 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:52.970+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:52.979 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:52.974+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:53.013 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:53.004+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:53.075 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:53.067+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:53.160 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:53.152+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:53.255 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:53.247+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:53.310 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:53.303+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:53.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:53.308+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:53.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:53.352+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:53.964 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:53.956+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:53.965 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:53.959+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:54.033 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:54.025+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:54.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:54.062+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:54.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:54.158+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:54.233 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:54.226+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:54.273 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:54.266+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:54.274 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:54.269+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:54.370 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:54.364+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:55.005 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:54.997+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:55.007 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:55.000+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:55.042 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:55.035+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:55.046 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:55.038+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:55.202 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:55.194+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:55.235 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:55.227+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:55.248 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:55.242+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:55.250 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:55.246+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:55.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:55.392+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:56.037 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:56.031+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:56.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:56.033+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:56.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:56.062+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:56.094 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:56.085+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:56.204 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:56.196+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:56.239 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:56.230+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:56.247 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:56.239+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:56.248 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:56.242+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:56.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:56.372+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:57.003 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:56.995+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:57.003 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:56.998+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:57.063 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:57.056+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:57.096 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:57.088+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:57.212 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:57.203+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:57.226 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:57.213+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:57.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:57.294+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:57.306 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:57.299+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:57.396 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:57.389+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:58.004 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:57.997+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:58.009 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:58.003+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:58.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:58.057+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:58.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:58.092+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:58.217 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:58.210+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:58.249 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:58.242+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:58.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:58.294+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:58.301 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:58.295+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:58.449 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:58.442+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:59.006 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:59.001+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:59.009 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:59.004+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:27:59.033 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:27:59.024+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:27:59.101 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:27:59.092+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:27:59.231 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:27:59.224+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:27:59.260 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:59.254+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:27:59.261 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:27:59.253+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:27:59.262 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:27:59.257+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:27:59.441 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:27:59.433+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:27:59.962 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:59.955+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:27:59.964 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:27:59.958+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:00.038 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:00.031+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:00.105 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:00.094+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:00.253 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:00.245+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:00.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:00.275+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:00.283 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:00.276+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:00.299 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:00.293+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:00.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:00.474+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:00.962 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:00.956+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:00.965 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:00.960+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:01.009 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:00.991+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:01.075 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:01.056+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:01.222 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:01.214+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:01.283 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:01.276+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:01.284 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:01.278+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:01.326 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:01.317+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:01.452 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:01.446+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:01.948 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:01.942+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:01.953 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:01.948+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:02.010 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:02.003+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:02.091 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:02.085+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:02.205 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:02.198+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:02.286 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:02.280+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:02.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:02.282+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:02.358 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:02.351+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:02.464 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:02.457+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:02.963 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:02.958+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:02.965 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:02.961+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:03.028 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:03.021+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:03.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:03.132+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:03.185 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:03.179+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:03.323 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:03.316+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:03.325 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:03.320+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:03.357 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:03.349+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:03.422 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:03.416+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:04.013 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:04.006+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:04.015 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:04.009+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:04.034 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:04.021+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:04.144 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:04.137+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:04.189 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:04.182+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:04.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:04.343+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:04.352 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:04.347+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:04.371 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:04.363+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:04.393 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:04.387+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:04.978 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:04.971+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:04.982 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:04.977+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:05.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:05.056+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:05.214 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:05.201+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:05.229 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:05.222+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:05.337 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:05.329+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:05.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:05.381+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:05.391 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:05.385+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:05.445 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:05.438+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:06.021 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:06.015+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:06.029 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:06.020+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:06.063 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:06.049+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:06.219 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:06.212+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:06.277 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:06.271+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:06.353 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:06.348+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:06.359 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:06.351+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:06.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:06.354+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:06.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:06.472+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:07.026 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:07.020+0000 1694c700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:07.056 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:07.049+0000 1694c700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.493046+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:07.059 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:07.053+0000 1694c700 -1 osd.0 313 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:07.233 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:07.225+0000 1694c700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:07.320 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:07.310+0000 1694c700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:07.326 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:07.318+0000 1694c700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:07.326 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:07.320+0000 1694c700 -1 osd.3 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:07.361 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:07.353+0000 1694c700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:07.467 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:07.461+0000 1694c700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:08.064 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:08.046+0000 1694c700 -1 osd.1 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~742 in=742b] snapc 0=[] RETRY=22 ondisk+retry+write+known_if_redirected e310) 2019-11-29T17:28:08.077 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:08.070+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.686440+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:08.232 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:08.226+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:08.251 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:08.196+0000 1694c700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:28:07.964646+0000 front 2019-11-29T17:27:05.487151+0000 (oldest deadline 2019-11-29T17:27:45.453373+0000) 2019-11-29T17:28:08.271 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:08.247+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:08.333 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:08.326+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:27:04.842893+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:08.334 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:08.328+0000 1694c700 -1 osd.3 314 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:104175 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5440973200] snapc 0=[] ondisk+write+known_if_redirected e313) 2019-11-29T17:28:08.394 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:08.354+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:08.401 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:08.347+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:27:04.904465+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:08.505 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:08.499+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:09.120 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:28:09.113+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:28:08.823963+0000 front 2019-11-29T17:27:05.685050+0000 (oldest deadline 2019-11-29T17:27:45.633562+0000) 2019-11-29T17:28:09.244 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:09.164+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:09.252 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:28:09.244+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:03.307958+0000 front 2019-11-29T17:27:05.658476+0000 (oldest deadline 2019-11-29T17:27:46.678176+0000) 2019-11-29T17:28:09.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:09.254+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:09.347 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:28:09.340+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.494800+0000 front 2019-11-29T17:28:08.474090+0000 (oldest deadline 2019-11-29T17:27:49.508861+0000) 2019-11-29T17:28:09.413 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:09.406+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:09.464 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:28:09.457+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.506448+0000 front 2019-11-29T17:28:09.061088+0000 (oldest deadline 2019-11-29T17:27:46.534633+0000) 2019-11-29T17:28:09.515 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:28:09.509+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:04.441199+0000 front 2019-11-29T17:27:04.443260+0000 (oldest deadline 2019-11-29T17:27:45.371215+0000) 2019-11-29T17:28:10.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:10.164+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:10.295 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:10.287+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:10.453 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:10.446+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:11.175 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:11.169+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:11.275 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:11.269+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:11.448 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:28:11.440+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:27:05.653858+0000 front 2019-11-29T17:27:05.685020+0000 (oldest deadline 2019-11-29T17:27:50.212534+0000) 2019-11-29T17:28:12.159 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:12.153+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:12.243 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:12.236+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:13.190 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:13.184+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:13.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:13.283+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:14.158 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:14.152+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:14.245 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:14.237+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:15.153 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:15.143+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:15.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:15.203+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:16.201 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:16.196+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:16.214 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:16.207+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:17.170 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:17.163+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:17.197 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:17.191+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:18.143 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:18.136+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:18.207 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:18.201+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:19.102 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:19.096+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:19.171 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:19.166+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:20.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:20.095+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:20.183 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:20.174+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:21.104 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:21.097+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:21.235 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:21.229+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:22.157 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:22.141+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:22.204 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:22.198+0000 1694c700 -1 osd.1 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104095 2.39s0 2:9e5a35d8:::42bb32be-1e05-491f-815b-a606bdf24d77.4355.611_%e3%9c%86%e1%a0%84%e0%a4%81%e3%9c%83%e5%98%87%e5%98%88%e9%90%8a%ed%88%8e%ed%88%8e%e3%9c%84%ef%84%8f%ef%84%8f%e9%90%8a%ef%84%92%ce%a9:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e310) 2019-11-29T17:28:22.820 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:28:22.809+0000 38279700 -1 osd.1 314 build_incremental_map_msg missing incremental map 312 2019-11-29T17:28:23.180 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:23.173+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:24.141 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:24.134+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:25.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:25.168+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:26.188 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:26.182+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:27.195 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:27.189+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:28.183 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:28.179+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:29.141 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:29.135+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:30.138 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:30.132+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:31.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:31.159+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:32.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:32.253+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:33.224 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:33.219+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:34.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:34.257+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:35.280 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:35.275+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:36.308 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:36.302+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:37.310 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:37.304+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:38.302 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:38.297+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:39.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:39.662+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:41.120 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:40.876+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:42.524 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:42.460+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:28:43.893 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:28:43.820+0000 1694c700 -1 osd.0 314 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101201 7.9 7:9a597dde:::2019-11-29-16-42bb32be-1e05-491f-815b-a606bdf24d77.4355.618-%e1%a0%84%e9%90%89%eb%8c%8e%e5%98%85%ed%88%90%e8%94%8b%e1%a0%83%e0%a4%81%e3%9c%83%e5%98%87%eb%8c%8d%ed%88%90%e0%a4%80%eb%8c%8d%e3%9c%86:head [append 0~737 in=737b] snapc 0=[] RETRY=24 ondisk+retry+write+known_if_redirected e312) 2019-11-29T17:29:35.514 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:35.509+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:36.563 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:36.556+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:36.618 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:36.611+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:36.756 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:36.748+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:36.896 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:36.887+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:37.287 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:37.280+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:37.362 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:37.355+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:37.524 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:37.516+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:37.646 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:37.638+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:37.750 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:37.744+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:37.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:37.839+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:37.891 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:37.883+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:38.298 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:38.290+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:38.410 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:38.402+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:38.489 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:38.481+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:38.636 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:38.628+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:38.724 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:38.717+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:38.858 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:38.850+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:38.882 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:38.875+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:39.268 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:39.259+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:39.430 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:39.423+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:39.541 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:39.534+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:39.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:39.642+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:39.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:39.730+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:39.830 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:39.822+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:39.862 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:39.855+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:40.272 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:40.263+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:40.404 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:40.396+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:40.579 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:40.573+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:40.662 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:40.649+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:40.768 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:40.762+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:40.866 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:40.857+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:40.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:40.867+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:41.282 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:41.274+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:41.385 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:41.379+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:41.557 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:41.549+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:41.700 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:41.693+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:41.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:41.769+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:41.914 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:41.907+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:41.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:41.911+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:42.312 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:42.304+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:42.363 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:42.356+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:42.595 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:42.589+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:42.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:42.703+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:42.807 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:42.799+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:42.945 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:42.936+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:42.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:42.945+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:43.359 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:43.352+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:43.372 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:43.364+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:43.610 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:43.604+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:43.672 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:43.664+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:43.837 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:43.830+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:43.906 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:43.899+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:43.989 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:43.967+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:44.341 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:44.316+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:44.391 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:44.383+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:44.574 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:44.566+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:44.627 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:44.619+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:44.803 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:44.791+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:44.905 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:44.898+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:44.974 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:44.968+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:45.340 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:45.331+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:45.433 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:45.424+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:45.592 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:45.584+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:45.597 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:45.588+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:45.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:45.754+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:45.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:45.909+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:45.951 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:45.945+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:46.321 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:46.312+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:46.388 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:46.380+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:46.565 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:46.559+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:46.648 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:46.640+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:46.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:46.731+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:46.946 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:46.938+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:46.996 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:46.988+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:47.316 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:47.308+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:47.411 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:47.403+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:47.521 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:47.514+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:47.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:47.650+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:47.725 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:47.718+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:47.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:47.933+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:47.972 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:47.965+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:48.328 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:48.319+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:48.430 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:48.422+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:48.515 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:48.509+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:48.626 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:48.618+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:48.766 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:48.759+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:48.976 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:48.968+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:49.024 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:49.016+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:49.319 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:49.311+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:49.476 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:49.468+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:49.496 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:49.489+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:49.657 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:49.649+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:49.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:49.727+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:50.029 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:50.012+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:50.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:50.062+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:50.296 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:50.287+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:50.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:50.500+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:50.538 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:50.531+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:50.633 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:50.624+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:50.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:50.750+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:51.035 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:51.026+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:51.082 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:51.075+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:51.354 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:51.340+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:51.468 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:51.460+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:51.506 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:51.498+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:51.608 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:51.600+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:51.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:51.783+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:52.021 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:52.013+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:52.055 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:52.047+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:52.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:52.364+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:52.504 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:52.496+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:52.520 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:52.512+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:52.636 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:52.627+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:52.844 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:52.835+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:53.003 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:52.992+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:53.083 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:53.076+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:53.375 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:53.368+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:53.511 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:53.504+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:53.577 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:53.564+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:53.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:53.650+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:53.866 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:53.859+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:53.967 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:53.959+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:54.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:54.110+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:54.414 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:54.407+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:54.562 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:54.555+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:54.568 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:54.562+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:54.643 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:54.637+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:54.871 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:54.864+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:54.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:54.967+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:55.091 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:55.083+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:55.438 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:55.431+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:55.533 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:55.525+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:55.548 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:55.541+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:55.681 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:55.674+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:55.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:55.894+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:55.955 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:55.948+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:56.108 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:56.101+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:56.422 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:56.414+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:56.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:56.494+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:56.591 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:56.583+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:56.691 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:56.684+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:56.913 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:56.908+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:57.012 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:57.004+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:57.158 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:57.152+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:57.409 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:57.399+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:57.476 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:57.469+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:57.584 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:57.577+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:57.721 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:57.713+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:57.939 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:57.932+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:57.986 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:57.979+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:58.207 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:58.202+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:58.439 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:58.433+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:58.513 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:58.500+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:58.577 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:58.572+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:58.690 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:58.682+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:58.922 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:58.916+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:29:59.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:29:59.030+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:29:59.206 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:29:59.199+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:29:59.442 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:29:59.435+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:29:59.503 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:29:59.497+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:29:59.629 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:29:59.622+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:29:59.709 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:29:59.702+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:29:59.946 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:29:59.940+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:00.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:00.007+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:00.237 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:00.231+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:00.433 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:00.426+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:00.533 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:00.526+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:00.658 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:00.653+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:00.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:00.725+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:00.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:00.912+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:01.052 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:01.044+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:01.202 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:01.196+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:01.429 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:01.421+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:01.563 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:01.556+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:01.635 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:01.630+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:01.771 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:01.765+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:01.954 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:01.947+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:02.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:02.003+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:02.222 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:02.216+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:02.486 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:02.477+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:02.573 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:02.565+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:02.661 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:02.656+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:02.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:02.794+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:02.928 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:02.922+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:03.032 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:03.024+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:03.203 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:03.195+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:03.513 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:03.505+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:03.542 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:03.536+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:03.698 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:03.693+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:03.758 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:03.750+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:03.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:03.945+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:04.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:04.011+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:04.181 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:04.168+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:04.514 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:04.507+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:04.567 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:04.559+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:04.701 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:04.695+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:04.782 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:04.776+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:04.943 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:04.938+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:05.064 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:05.057+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:05.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:05.132+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:05.475 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:05.468+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:05.527 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:05.520+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:05.703 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:05.697+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:05.808 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:05.802+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:05.911 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:05.904+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:06.021 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:06.014+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:06.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:06.142+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:06.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:06.479+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:06.524 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:06.513+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:06.691 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:06.685+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:06.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:06.759+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:06.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:06.924+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:07.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:07.006+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:07.154 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:07.148+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:07.500 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:07.481+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:07.556 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:07.549+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:07.721 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:07.713+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:07.792 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:07.785+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:07.910 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:07.905+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:08.060 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:08.052+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:08.126 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:08.120+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:08.488 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:08.481+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:08.563 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:08.555+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:08.742 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:30:08.736+0000 1694c700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:50.809894+0000 front 2019-11-29T17:28:50.941748+0000 (oldest deadline 2019-11-29T17:29:35.077289+0000) 2019-11-29T17:30:08.852 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:30:08.845+0000 1694c700 -1 osd.4 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:52.642590+0000 front 2019-11-29T17:28:54.103547+0000 (oldest deadline 2019-11-29T17:29:36.473251+0000) 2019-11-29T17:30:08.882 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:30:08.876+0000 1694c700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.526376+0000 front 2019-11-29T17:28:51.538381+0000 (oldest deadline 2019-11-29T17:29:36.019036+0000) 2019-11-29T17:30:09.042 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:30:09.034+0000 1694c700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.077061+0000 front 2019-11-29T17:28:50.914673+0000 (oldest deadline 2019-11-29T17:29:36.724988+0000) 2019-11-29T17:30:09.116 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:30:09.110+0000 1694c700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.537579+0000 front 2019-11-29T17:28:54.200261+0000 (oldest deadline 2019-11-29T17:29:37.592788+0000) 2019-11-29T17:30:09.494 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:30:09.480+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 46sec e314 v314) 2019-11-29T17:30:09.540 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:30:09.532+0000 1694c700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:51.540800+0000 front 2019-11-29T17:28:54.104560+0000 (oldest deadline 2019-11-29T17:29:36.737526+0000) 2019-11-29T17:30:09.560 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:30:09.554+0000 1694c700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:28:54.831107+0000 front 2019-11-29T17:28:54.791951+0000 (oldest deadline 2019-11-29T17:29:37.034198+0000) 2019-11-29T17:30:24.769 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:17:37.656 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:24.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:17:37.696 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:24.840 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:17:37.614 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:24.843 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:17:37.616 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:24.845 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:17:37.733 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:24.846 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:17:37.734 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:25.019 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:17:37.906 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:25.033 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:17:37.921 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:25.072 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:17:37.845 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:30:39.713 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:30:39.697+0000 28259700 -1 reset not still connected to 0xeac6730 2019-11-29T17:30:39.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:30:39.698+0000 28259700 -1 reset not still connected to 0x100e4eb0 2019-11-29T17:31:07.960 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:07.951+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:08.662 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:08.654+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:08.931 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:08.924+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:09.090 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:09.077+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:09.646 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:09.639+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:09.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:09.928+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:10.094 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:10.086+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:10.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:10.651+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:10.688 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:10.681+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:10.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:10.892+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:11.090 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:11.083+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:11.642 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:11.636+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:11.688 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:11.681+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:11.906 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:31:11.867+0000 20e22700 -1 RGWWatcher::handle_error cookie 441179728 err (107) Transport endpoint is not connected 2019-11-29T17:31:11.933 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:11.926+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:12.079 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:12.072+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:12.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:12.589+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:12.664 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:12.655+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:12.668 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:12.659+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:12.914 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:12.907+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:13.084 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:13.078+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:13.564 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:13.557+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:13.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:13.641+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:13.682 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:13.675+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:13.925 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:13.918+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:14.069 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:14.061+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:14.576 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:14.570+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:14.615 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:14.608+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:14.637 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:14.632+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:14.703 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:14.696+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:14.943 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:14.935+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:15.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:15.036+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:15.558 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:15.552+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:15.635 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:15.628+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:15.659 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:15.654+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:15.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:15.690+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:15.964 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:15.957+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:16.096 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:16.088+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:16.577 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:16.570+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:16.650 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:16.645+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:16.654 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:16.644+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:16.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:16.680+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:16.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:16.928+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:17.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:17.057+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:17.603 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:17.596+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:17.655 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:17.649+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:17.696 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:17.690+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:17.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:17.721+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:17.901 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:17.895+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:18.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:18.034+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:18.690 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:18.684+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:18.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:18.684+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:18.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:18.689+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:18.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:18.717+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:18.782 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:18:31.670 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.789 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:18:31.677 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.795 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:18:31.682 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.796 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:18:31.569 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.799 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:18:31.687 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.800 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:18:31.573 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.883 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:18.875+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:18.885 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:18:31.658 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.888 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:18:31.775 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:18.896 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:18:31.784 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:31:19.028 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:19.022+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:19.684 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:19.678+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:19.695 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:19.689+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:19.721 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:19.715+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:19.752 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:19.745+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:19.891 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:19.885+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:20.017 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:20.010+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:20.684 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:20.678+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:20.691 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:20.685+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:20.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:20.698+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:20.741 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:20.733+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:20.886 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:20.879+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:21.004 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:20.997+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:21.650 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:21.642+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:21.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:21.649+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:21.660 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:21.654+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:21.694 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:21.686+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:21.697 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:21.691+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:21.929 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:21.922+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:22.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:22.001+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:22.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:22.641+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:22.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:22.680+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:22.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:22.682+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:22.717 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:22.709+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:22.737 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:22.730+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:22.960 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:22.953+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:22.965 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:22.957+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:23.677 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:23.670+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:23.707 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:23.700+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:23.733 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:23.726+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:23.734 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:23.728+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:23.768 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:23.761+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:23.937 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:23.930+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:23.983 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:23.976+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:24.696 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:24.689+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:24.717 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:24.708+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:24.721 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:24.715+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:24.742 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:24.735+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:24.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:24.736+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:24.909 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:24.898+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:24.939 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:24.932+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:25.694 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:25.686+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:25.710 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:25.702+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:25.717 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:25.709+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:25.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:25.711+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:25.723 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:25.715+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:25.873 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:25.866+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:25.918 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:25.905+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:26.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:26.685+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:26.695 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:26.689+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:26.697 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:26.690+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:26.703 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:26.695+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:26.733 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:26.726+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:26.864 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:26.857+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:26.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:26.892+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:27.680 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:27.674+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:27.682 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:27.675+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:27.708 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:27.702+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:27.709 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:27.703+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:27.710 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:27.705+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:27.864 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:27.857+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:27.930 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:27.923+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:28.639 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:28.632+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:28.692 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:28.685+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:28.716 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:28.710+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:28.735 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:28.730+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:28.736 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:28.731+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:28.871 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:28.864+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:28.907 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:28.900+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:29.642 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:29.636+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:29.650 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:29.643+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:29.684 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:29.678+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:29.780 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:29.772+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:29.781 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:29.775+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:29.922 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:29.914+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:29.923 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:29.916+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:30.639 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:30.632+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:30.664 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:30.657+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:30.723 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:30.716+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:30.804 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:30.798+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:30.806 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:30.802+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:30.892 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:30.884+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:30.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:30.911+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:31.639 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:31.632+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:31.640 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:31.634+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:31.759 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:31.753+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:31.838 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:31.831+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:31.839 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:31.834+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:31.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:31.848+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:31.878 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:31.873+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:32.601 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:32.594+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:32.639 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:32.634+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:32.732 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:32.726+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:32.826 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:32.818+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:32.827 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:32.821+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:32.880 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:32.873+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:32.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:32.910+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:33.559 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:33.552+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:33.650 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:33.643+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:33.715 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:33.708+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:33.852 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:33.844+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:33.881 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:33.874+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:33.883 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:33.877+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:33.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:33.924+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:34.586 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:34.579+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:34.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:34.662+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:34.683 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:34.677+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:34.879 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:34.873+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:34.908 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:34.902+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:34.910 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:34.904+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:34.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:34.941+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:35.604 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:35.598+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:35.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:35.625+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:35.673 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:35.666+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:35.861 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:35.856+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:35.862 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:35.857+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:35.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:35.888+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:35.905 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:35.901+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:36.587 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:36.580+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:36.636 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:36.629+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:36.699 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:36.692+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:36.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:36.858+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:36.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:36.859+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:36.914 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:36.902+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:36.932 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:36.926+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:37.245 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:31:37.241+0000 20e22700 -1 RGWWatcher::handle_error cookie 820696576 err (107) Transport endpoint is not connected 2019-11-29T17:31:37.577 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:37.571+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:37.666 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:37.658+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:37.753 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:37.744+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:37.845 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:37.839+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:37.851 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:37.846+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:37.944 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:37.933+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:37.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:37.962+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:38.557 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:38.551+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:38.657 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:38.648+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:38.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:38.735+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:38.842 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:38.836+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:38.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:38.838+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:38.958 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:38.950+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:38.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:38.969+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:39.608 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:39.602+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:39.693 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:39.685+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:39.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:39.735+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:39.887 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:39.881+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:39.892 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:39.887+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:39.925 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:39.918+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:39.960 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:39.954+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:39.961 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:39.956+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:40.570 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:40.563+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:40.706 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:40.698+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:40.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:40.759+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:40.885 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:40.877+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:40.887 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:40.881+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:40.951 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:40.943+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:40.963 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:40.954+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:40.963 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:40.959+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:41.572 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:41.567+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:41.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:41.719+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:41.746 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:41.739+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:41.912 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:41.905+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:41.919 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:41.913+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:41.924 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:41.919+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:41.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:41.961+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:41.970 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:41.966+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:42.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:42.595+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:42.748 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:42.740+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:42.749 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:42.741+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:42.904 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:42.895+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:42.962 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:42.955+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:42.963 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:42.957+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:43.014 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:43.009+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:43.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:43.012+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:43.634 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:43.628+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:43.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:43.702+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:43.752 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:43.744+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:43.937 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:43.926+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:43.992 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:43.986+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:43.994 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:43.990+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:44.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:44.034+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:44.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:44.038+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:44.618 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:44.612+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:44.719 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:44.710+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:44.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:44.745+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:44.896 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:44.888+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:44.975 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:44.968+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:44.976 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:44.970+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:45.064 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:31:45.057+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e319 v319) 2019-11-29T17:31:45.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:45.064+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:45.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:45.080+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:45.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:45.081+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:45.628 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:45.621+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:45.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:45.665+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:45.801 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:45.794+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:45.892 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:45.885+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:46.004 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:45.997+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:46.005 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:46.000+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:46.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:46.060+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:46.068 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:46.063+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:46.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:46.088+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:46.651 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:46.645+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:46.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:46.716+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:46.787 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:46.780+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:46.879 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:46.873+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:47.030 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:47.024+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:47.031 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:47.025+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:47.052 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:47.047+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:47.055 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:47.050+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:47.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:47.053+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:47.652 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:47.646+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:47.720 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:47.713+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:47.782 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:47.774+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:47.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:47.864+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:48.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:48.003+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:48.012 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:48.005+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:48.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:48.052+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:48.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:48.054+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:48.081 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:48.075+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:48.657 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:48.651+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:48.682 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:48.675+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:48.809 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:48.801+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:48.829 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:48.822+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:49.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:49.037+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:49.046 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:49.041+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:49.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:49.083+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:49.090 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:49.085+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:49.130 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:49.125+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:49.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:49.624+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:49.649 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:49.642+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:49.814 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:49.808+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:49.859 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:49.852+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:50.083 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:31:50.076+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e319 v319) 2019-11-29T17:31:50.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:50.081+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:50.089 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:50.083+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:50.092 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:50.082+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:50.094 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:50.088+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:50.150 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:50.144+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:50.613 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:50.606+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:50.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:50.643+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:50.836 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:50.830+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:50.842 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:50.836+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:51.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:51.053+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:51.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:51.055+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:51.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:51.079+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:51.090 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:51.084+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:51.128 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:51.122+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:51.570 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:51.564+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:51.672 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:51.666+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:51.846 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:51.838+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:51.874 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:51.867+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:52.057 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:52.049+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:52.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:52.054+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:52.093 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:52.087+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:52.096 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:52.089+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:52.119 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:52.114+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:52.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:52.600+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:52.683 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:52.677+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:52.809 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:52.802+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:52.880 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:52.872+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:53.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:53.082+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:53.091 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:53.087+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:53.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:53.094+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:53.103 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:53.098+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:53.124 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:53.118+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:53.607 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:53.600+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:53.647 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:53.640+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:53.847 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:53.840+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:53.852 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:53.844+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:54.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:54.056+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:54.067 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:54.060+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:54.111 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:54.104+0000 1694c700 -1 osd.6 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.230267+0000 front 2019-11-29T17:30:25.224259+0000 (oldest deadline 2019-11-29T17:31:53.948942+0000) 2019-11-29T17:31:54.114 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:54.109+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:54.116 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:54.107+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:54.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:54.110+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:54.605 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:31:54.597+0000 1694c700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.247674+0000 front 2019-11-29T17:30:26.246617+0000 (oldest deadline 2019-11-29T17:31:12.111843+0000) 2019-11-29T17:31:54.675 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:31:54.669+0000 1694c700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.436210+0000 front 2019-11-29T17:30:24.449930+0000 (oldest deadline 2019-11-29T17:31:08.509732+0000) 2019-11-29T17:31:54.839 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:31:54.833+0000 1694c700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.068501+0000 front 2019-11-29T17:30:25.066942+0000 (oldest deadline 2019-11-29T17:31:10.286350+0000) 2019-11-29T17:31:54.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:31:54.863+0000 1694c700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:24.986850+0000 front 2019-11-29T17:30:24.985985+0000 (oldest deadline 2019-11-29T17:31:07.238268+0000) 2019-11-29T17:31:55.050 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:55.040+0000 1694c700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:26.781693+0000 front 2019-11-29T17:30:26.588876+0000 (oldest deadline 2019-11-29T17:31:08.117827+0000) 2019-11-29T17:31:55.050 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:55.042+0000 1694c700 -1 osd.3 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:55.085 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:55.072+0000 1694c700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:30:40.850703+0000 (oldest deadline 2019-11-29T17:31:20.850703+0000) 2019-11-29T17:31:55.085 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:31:55.075+0000 1694c700 -1 osd.0 319 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:101386 7.9 7.7bbe9a59 (undecoded) ondisk+retry+write+known_if_redirected e318) 2019-11-29T17:31:55.122 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:31:55.115+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e319 v319) 2019-11-29T17:31:55.153 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:55.145+0000 1694c700 -1 osd.6 319 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:30:25.230267+0000 front 2019-11-29T17:30:25.224259+0000 (oldest deadline 2019-11-29T17:31:53.948942+0000) 2019-11-29T17:31:55.154 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:55.149+0000 1694c700 -1 osd.6 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:56.052 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:31:56.047+0000 1694c700 -1 osd.3 320 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:104763 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 820696576] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:56.112 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:56.107+0000 1694c700 -1 osd.6 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:31:57.179 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:31:57.173+0000 1694c700 -1 osd.6 321 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104769 7.d 7:b564efa4:usage::usage.2:head [call rgw.user_usage_log_add in=212b] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-11-29T17:32:15.193 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:32:15.142+0000 1694c700 -1 osd.1 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:104784 6.1 6.c9be1e91 (undecoded) ondisk+read+known_if_redirected e319) 2019-11-29T17:33:41.615 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:41.608+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:42.322 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:42.311+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:42.570 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:42.563+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:42.970 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:42.962+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:43.279 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:43.271+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:43.532 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:43.524+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:43.981 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:43.973+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:44.277 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:44.270+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:44.330 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:44.322+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:44.538 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:44.529+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:44.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:44.889+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:44.950 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:44.938+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:45.330 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:45.323+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:45.331 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:45.326+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:45.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:45.511+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:45.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:45.869+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:45.921 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:45.914+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:46.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:46.298+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:46.382 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:46.375+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:46.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:46.509+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:46.859 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:46.852+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:46.965 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:46.957+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:47.099 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:47.090+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:47.184 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:47.177+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:47.337 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:47.330+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:47.370 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:47.363+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:47.507 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:47.500+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:47.896 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:47.890+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:47.957 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:47.950+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:48.074 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:48.067+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:48.183 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:48.176+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:48.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:48.322+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:48.344 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:48.337+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:48.493 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:48.486+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:48.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:48.877+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:48.940 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:48.933+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:49.091 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:49.083+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:49.224 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:49.218+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:49.289 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:49.282+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:49.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:49.381+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:49.488 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:49.483+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:49.892 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:49.885+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:49.982 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:49.975+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:50.118 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:50.110+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:50.245 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:50.239+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:50.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:50.259+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:50.369 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:50.357+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:50.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:50.510+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:50.854 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:50.847+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:51.014 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:51.007+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:51.109 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:51.102+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:51.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:51.225+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:51.268 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:51.262+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:51.362 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:51.354+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:51.522 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:51.517+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:51.895 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:51.887+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:52.034 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:52.027+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:52.132 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:52.125+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:52.237 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:52.231+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:52.299 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:52.291+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:52.397 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:52.390+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:52.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:52.516+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:52.914 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:52.906+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:53.037 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:53.029+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:53.154 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:53.147+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:53.290 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:53.283+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:53.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:53.286+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:53.365 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:53.357+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:53.488 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:53.482+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:53.912 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:53.906+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:54.054 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:54.047+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:54.194 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:54.186+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:54.252 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:54.246+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:54.336 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:54.330+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:54.414 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:54.406+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:54.508 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:54.502+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:54.967 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:54.960+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:55.068 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:55.060+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:55.155 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:55.148+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:55.283 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:55.276+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:55.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:55.343+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:55.433 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:55.427+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:55.509 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:55.504+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:55.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:55.968+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:56.029 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:56.022+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:56.145 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:56.130+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:56.274 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:56.268+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:56.392 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:56.385+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:56.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:56.426+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:56.532 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:56.525+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:56.949 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:56.941+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:57.000 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:56.994+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:57.196 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:57.187+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:57.314 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:57.307+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:57.413 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:57.405+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:57.443 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:57.436+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:57.565 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:57.559+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:57.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:57.955+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:58.041 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:58.029+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:58.219 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:58.212+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:58.327 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:58.322+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:58.446 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:58.439+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:58.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:58.460+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:58.593 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:58.587+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:58.988 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:58.981+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:59.010 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:59.004+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:33:59.196 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:33:59.188+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:33:59.326 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:33:59.320+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:33:59.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:33:59.418+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:33:59.461 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:33:59.454+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:33:59.581 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:33:59.575+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:33:59.952 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:33:59.945+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:33:59.996 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:33:59.990+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:00.229 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:00.222+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:00.302 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:00.297+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:00.469 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:00.462+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:00.490 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:00.483+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:00.629 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:00.623+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:00.962 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:00.956+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:00.972 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:00.965+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:01.185 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:01.177+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:01.272 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:01.264+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:01.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:01.506+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:01.522 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:01.516+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:01.667 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:01.661+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:01.930 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:01.923+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:01.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:01.989+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:02.177 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:02.170+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:02.299 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:02.292+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:02.470 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:02.462+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:02.489 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:02.483+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:02.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:02.663+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:02.951 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:02.944+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:03.045 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:03.037+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:03.216 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:03.208+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:03.317 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:03.310+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:03.435 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:03.427+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:03.472 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:03.465+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:03.657 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:03.651+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:03.909 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:03.902+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:04.057 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:04.050+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:04.232 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:04.225+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:04.273 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:04.267+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:04.431 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:04.424+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:04.460 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:04.453+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:04.692 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:04.687+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:04.903 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:04.896+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:05.101 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:05.094+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:05.254 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:05.248+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:05.279 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:05.272+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:05.462 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:05.455+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:05.493 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:05.487+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:05.717 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:05.710+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:05.906 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:05.898+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:06.114 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:06.107+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:06.248 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:06.242+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:06.294 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:06.287+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:06.489 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:06.481+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:06.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:06.519+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:06.701 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:06.694+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:06.874 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:06.867+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:07.081 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:07.073+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:07.206 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:07.198+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:07.304 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:07.297+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:07.446 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:07.439+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:07.497 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:07.491+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:07.717 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:07.710+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:07.873 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:07.866+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:08.134 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:08.126+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:08.169 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:08.162+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:08.309 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:08.301+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:08.441 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:08.433+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:08.550 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:08.544+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:08.747 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:08.742+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:08.914 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:08.905+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:09.115 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:09.107+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:09.127 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:09.120+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:09.359 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:09.351+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:09.403 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:09.394+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:09.512 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:09.505+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:09.725 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:09.719+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:09.911 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:09.903+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:10.125 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:10.115+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:10.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:10.157+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:10.354 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:10.345+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:10.365 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:10.357+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:10.555 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:10.548+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:10.727 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:10.719+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:10.878 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:10.870+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:11.129 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:11.121+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:11.157 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:11.148+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:11.333 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:11.326+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:11.336 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:11.328+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:11.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:11.517+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:11.777 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:11.769+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:11.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:11.847+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:12.128 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:12.121+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:12.191 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:12.183+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:12.295 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:12.285+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:12.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:12.381+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:12.539 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:12.531+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:12.795 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:12.786+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:12.884 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:12.874+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:13.107 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:13.101+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:13.199 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:13.193+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:13.261 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:13.254+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:13.389 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:13.382+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:13.546 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:13.540+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:13.758 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:13.750+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:13.862 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:13.854+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:14.063 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:14.056+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:14.216 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:14.193+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:14.308 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:14.300+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:14.371 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:14.364+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:14.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:14.530+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:14.720 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:14.714+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:14.863 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:14.856+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:15.093 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:15.086+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:15.172 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:15.163+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:15.295 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:15.286+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:15.344 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:15.336+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:15.485 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:34:15.479+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e324 v324) 2019-11-29T17:34:15.546 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:15.539+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:15.699 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:15.693+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:15.845 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:15.823+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:16.121 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:16.114+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:16.192 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:16.180+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:16.279 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:16.271+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:16.333 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:16.323+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:16.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:16.531+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:16.734 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:16.728+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:16.817 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:16.809+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:17.155 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:17.148+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:17.236 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:17.229+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:17.238 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:17.229+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:17.351 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:17.342+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:17.502 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:17.495+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:17.692 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:17.684+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:17.845 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:17.836+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:18.192 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:18.184+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:18.234 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:18.226+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:18.245 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:18.238+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:18.339 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:18.333+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:18.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:18.489+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:18.687 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:18.676+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:18.874 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:18.867+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:19.153 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:19.139+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:19.241 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:19.235+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:19.276 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:19.269+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:19.302 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:19.295+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:19.505 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:19.491+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:19.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:34:19.690+0000 1694c700 -1 osd.2 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:00.897119+0000 front 2019-11-29T17:32:57.370844+0000 (oldest deadline 2019-11-29T17:33:40.870794+0000) 2019-11-29T17:34:19.861 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:34:19.845+0000 1694c700 -1 osd.7 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:57.562268+0000 front 2019-11-29T17:32:57.514240+0000 (oldest deadline 2019-11-29T17:33:42.780763+0000) 2019-11-29T17:34:20.159 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:34:20.151+0000 1694c700 -1 osd.1 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.152752+0000 front 2019-11-29T17:33:01.679230+0000 (oldest deadline 2019-11-29T17:33:46.426315+0000) 2019-11-29T17:34:20.214 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:20.204+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:20.269 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:34:20.262+0000 1694c700 -1 osd.4 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.275577+0000 front 2019-11-29T17:32:59.298008+0000 (oldest deadline 2019-11-29T17:33:41.460317+0000) 2019-11-29T17:34:20.316 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:34:20.308+0000 1694c700 -1 osd.6 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.680654+0000 front 2019-11-29T17:33:01.156324+0000 (oldest deadline 2019-11-29T17:33:46.444249+0000) 2019-11-29T17:34:20.516 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:34:20.508+0000 1694c700 -1 osd.3 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:32:59.070519+0000 front 2019-11-29T17:32:58.998080+0000 (oldest deadline 2019-11-29T17:33:43.677908+0000) 2019-11-29T17:34:20.535 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:34:20.527+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e324 v324) 2019-11-29T17:34:21.204 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:34:21.198+0000 1694c700 -1 osd.5 324 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:33:01.897385+0000 front 2019-11-29T17:33:01.651182+0000 (oldest deadline 2019-11-29T17:33:44.482377+0000) 2019-11-29T17:34:49.976 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:34:49.969+0000 3a27d700 -1 osd.0 330 build_incremental_map_msg missing incremental map 330 2019-11-29T17:34:51.903 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:34:51.881+0000 3a27d700 -1 osd.0 330 build_incremental_map_msg missing incremental map 330 2019-11-29T17:35:14.973 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:22:27.861 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:14.978 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:22:27.866 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:14.985 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:22:27.873 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:14.990 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:22:27.878 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:14.996 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:22:27.770 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:15.000 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:22:27.773 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:15.315 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:22:28.203 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:15.343 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:22:28.116 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:15.345 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:22:28.233 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:35:16.612 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:35:16.502+0000 20e22700 -1 RGWWatcher::handle_error cookie 5475958320 err (107) Transport endpoint is not connected 2019-11-29T17:35:18.245 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:18.238+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:18.251 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:18.244+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:19.253 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:19.247+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:19.259 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:19.251+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:19.345 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:19.341+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:19.704 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:19.697+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:20.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:20.066+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:20.255 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:20.250+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:20.303 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:20.297+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:20.352 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:20.348+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:20.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:20.718+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:20.728 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:20.722+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:21.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:21.055+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:21.212 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:21.206+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:21.310 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:21.303+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:21.402 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:21.397+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:21.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:21.748+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:21.758 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:21.753+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:21.910 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:21.903+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:22.050 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:22.042+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:22.181 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:22.174+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:22.363 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:22.356+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:22.446 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:22.441+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:22.714 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:22.706+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:22.714 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:22.707+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:22.896 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:22.889+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:22.960 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:22.951+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:23.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:23.088+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:23.178 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:23.172+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:23.361 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:23.354+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:23.475 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:23.470+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:23.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:23.729+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:23.736 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:23.731+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:23.857 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:23.851+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:23.976 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:23.969+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:24.083 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:24.076+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:24.143 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:24.137+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:24.387 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:24.379+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:24.444 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:24.439+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:24.691 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:24.686+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:24.692 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:24.688+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:24.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:24.814+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:24.939 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:24.931+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:25.058 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:25.051+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:25.164 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:25.158+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:25.409 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:25.404+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:25.422 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:25.414+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:25.657 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:25.649+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:25.658 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:25.652+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:25.847 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:25.840+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:25.958 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:25.950+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:26.106 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:26.098+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:26.213 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:26.206+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:26.397 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:26.390+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:26.398 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:26.393+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:26.433 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:26.424+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:26.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:26.629+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:26.638 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:26.632+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:26.878 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:26.871+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:27.008 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:26.997+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:27.113 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:27.105+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:27.220 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:27.214+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:27.409 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:27.401+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:27.426 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:27.420+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:27.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:27.424+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:27.637 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:27.631+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:27.638 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:27.633+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:27.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:27.923+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:27.992 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:27.985+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:28.090 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:28.083+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:28.199 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:28.193+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:28.398 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:28.391+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:28.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:28.396+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:28.415 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:28.408+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:28.644 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:28.637+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:28.646 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:28.642+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:28.917 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:28.908+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:28.958 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:28.950+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:29.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:29.043+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:29.223 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:29.217+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:29.450 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:29.444+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:29.454 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:29.448+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:29.457 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:29.449+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:29.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:29.630+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:29.637 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:29.632+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:29.922 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:29.915+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:29.926 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:29.918+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:30.011 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:30.003+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:30.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:30.205+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:30.433 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:30.426+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:30.435 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:30.429+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:30.504 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:30.496+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:30.648 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:30.642+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:30.649 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:30.644+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:30.903 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:30.896+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:30.935 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:30.928+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:31.037 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:31.029+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:31.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:31.205+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:31.441 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:31.434+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:31.567 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:31.497+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:31.567 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:31.437+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:31.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:31.668+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:31.676 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:31.671+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:31.872 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:31.864+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:31.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:31.943+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:32.084 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:32.077+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:32.170 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:32.164+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:32.467 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:32.459+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:32.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:32.462+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:32.516 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:32.507+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:32.661 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:32.654+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:32.663 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:32.656+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:32.836 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:32.829+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:32.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:32.938+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:33.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:33.110+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:33.197 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:33.191+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:33.480 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:33.471+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:33.491 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:33.484+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:33.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:33.490+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:33.668 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:33.660+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:33.669 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:33.663+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:33.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:33.803+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:33.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:33.956+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:34.132 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:34.124+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:34.154 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:34.149+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:34.486 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:34.479+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:34.488 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:34.482+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:34.524 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:34.517+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:34.681 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:34.672+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:34.681 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:34.673+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:34.829 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:34.823+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:34.925 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:34.919+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:35.123 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:35.116+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:35.204 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:35.197+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:35.481 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:35.475+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:35.483 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:35.477+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:35.574 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:35.567+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:35.684 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:35.677+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:35.685 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:35.680+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:35.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:35.833+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:35.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:35.956+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:36.093 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:36.086+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:36.164 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:36.157+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:36.468 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:36.462+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:36.471 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:36.465+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:36.603 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:36.592+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:36.641 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:36.635+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:36.646 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:36.639+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:36.802 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:36.795+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:36.923 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:36.917+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:37.111 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:37.104+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:37.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:37.186+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:37.461 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:37.454+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:37.462 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:37.457+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:37.626 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:37.621+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:37.629 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:37.624+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:37.654 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:37.647+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:37.804 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:37.798+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:37.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:37.947+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:38.089 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:38.082+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:38.222 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:38.215+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:38.440 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:38.435+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:38.443 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:38.439+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:38.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:38.594+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:38.602 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:38.596+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:38.618 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:38.610+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:38.820 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:38.813+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:38.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:38.959+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:39.143 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:39.136+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:39.202 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:39.196+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:39.458 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:39.448+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:39.460 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:39.453+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:39.618 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:39.611+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:39.635 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:39.627+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:39.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:39.631+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:39.823 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:39.817+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:39.998 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:39.991+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:40.180 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:40.173+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:40.183 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:40.175+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:40.418 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:40.410+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:40.418 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:40.413+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:40.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:40.575+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:40.685 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:40.679+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:40.686 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:40.681+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:40.801 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:40.795+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:41.008 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:41.000+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:41.152 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:41.145+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:41.184 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:41.177+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:41.379 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:41.371+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:41.380 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:41.374+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:41.620 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:41.612+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:41.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:41.712+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:41.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:41.713+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:41.855 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:41.848+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:41.987 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:41.980+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:42.138 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:42.131+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:42.230 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:42.223+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:42.433 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:42.425+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:42.434 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:42.427+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:42.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:42.576+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:42.745 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:42.738+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:42.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:42.740+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:42.834 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:42.826+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:42.946 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:42.938+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:43.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:43.136+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:43.278 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:43.270+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:43.417 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:43.411+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:43.419 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:43.414+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:43.592 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:43.585+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:43.765 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:43.757+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:43.766 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:43.760+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:43.887 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:43.879+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:43.981 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:43.974+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:44.148 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:44.141+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:44.256 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:44.248+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:44.450 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:44.442+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:44.451 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:44.445+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:44.608 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:44.601+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:44.730 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:44.722+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:44.731 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:44.724+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:44.914 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:44.907+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:44.982 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:44.976+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:45.119 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:45.112+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:45.221 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:45.215+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:45.481 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:45.474+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:45.485 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:45.476+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:45.644 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:45.638+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:45.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:45.770+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:45.779 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:45.773+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:45.946 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:45.939+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:45.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:45.989+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:46.106 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:46.101+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:46.232 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:46.226+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:46.489 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:46.482+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:46.494 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:46.487+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:46.688 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:46.680+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:46.754 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:46.746+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:46.756 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:46.749+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:46.910 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:46.903+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:46.986 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:46.979+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:47.159 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:47.152+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:47.199 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:47.194+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:47.515 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:47.508+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:47.517 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:47.509+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:47.675 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:47.669+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:47.781 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:47.773+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:47.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:47.775+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:47.944 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:47.936+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:47.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:47.957+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:48.186 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:48.178+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:48.231 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:48.224+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:48.530 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:48.523+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:48.533 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:48.528+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:48.708 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:48.699+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:48.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:48.750+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:48.759 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:48.753+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:48.915 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:48.908+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:48.945 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:48.938+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:49.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:49.135+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:49.197 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:49.188+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:49.528 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:49.519+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:49.529 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:49.522+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:49.695 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:49.689+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:49.779 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:49.769+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:49.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:49.775+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:49.896 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:49.888+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:49.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:49.944+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:50.140 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:50.132+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:50.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:50.184+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:50.554 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:50.546+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:50.557 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:50.549+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:50.744 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:50.737+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:50.744 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:50.739+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:50.749 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:50.744+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:50.864 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:35:50.857+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e330 v330) 2019-11-29T17:35:50.898 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:50.891+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:50.916 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:50.909+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:51.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:51.151+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:51.215 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:51.208+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:51.550 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:51.543+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:51.554 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:51.548+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:51.745 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:51.737+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:51.752 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:51.743+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:51.753 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:51.747+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:51.901 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:51.894+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:51.931 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:51.924+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:52.141 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:52.134+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:52.203 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:52.197+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:52.518 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:52.511+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:52.523 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:52.516+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:52.739 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:52.734+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:52.742 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:52.737+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:52.791 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:52.784+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:52.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:52.878+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:52.911 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:52.904+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:53.178 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:53.172+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:53.193 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:53.185+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:53.576 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:53.567+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:53.576 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:53.569+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:53.756 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:53.747+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:53.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:53.750+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:53.780 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:53.773+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:53.889 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:53.882+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:53.923 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:53.916+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:54.188 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:54.180+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:54.211 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:54.201+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:54.534 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:54.526+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:54.536 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:54.529+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:54.791 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:54.784+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:54.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:54.785+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:54.795 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:54.789+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:54.858 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:54.851+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:54.936 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:54.930+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:55.195 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:35:55.188+0000 1694c700 -1 osd.7 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.617111+0000 front 2019-11-29T17:34:36.635421+0000 (oldest deadline 2019-11-29T17:35:19.483628+0000) 2019-11-29T17:35:55.202 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:35:55.195+0000 1694c700 -1 osd.2 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.259144+0000 front 2019-11-29T17:34:32.793916+0000 (oldest deadline 2019-11-29T17:35:17.220151+0000) 2019-11-29T17:35:55.494 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:55.485+0000 1694c700 -1 osd.0 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:34:45.549272+0000 (oldest deadline 2019-11-29T17:35:25.549272+0000) 2019-11-29T17:35:55.496 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:35:55.489+0000 1694c700 -1 osd.0 330 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:105194 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e329) 2019-11-29T17:35:55.767 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:35:55.756+0000 1694c700 -1 osd.4 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:33.275317+0000 front 2019-11-29T17:34:37.922411+0000 (oldest deadline 2019-11-29T17:35:17.848037+0000) 2019-11-29T17:35:55.774 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:55.767+0000 1694c700 -1 osd.3 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.089183+0000 front 2019-11-29T17:34:36.065866+0000 (oldest deadline 2019-11-29T17:35:18.882043+0000) 2019-11-29T17:35:55.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:55.772+0000 1694c700 -1 osd.3 330 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:55.815 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:35:55.808+0000 1694c700 -1 osd.5 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:36.316478+0000 front 2019-11-29T17:34:36.333371+0000 (oldest deadline 2019-11-29T17:35:21.947700+0000) 2019-11-29T17:35:55.904 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:35:55.898+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e330 v330) 2019-11-29T17:35:55.916 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:35:55.908+0000 1694c700 -1 osd.6 330 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:34:37.143414+0000 front 2019-11-29T17:34:37.216702+0000 (oldest deadline 2019-11-29T17:35:21.217045+0000) 2019-11-29T17:35:56.759 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:56.753+0000 1694c700 -1 osd.3 331 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105199 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5475958320] snapc 0=[] ondisk+write+known_if_redirected e329) 2019-11-29T17:35:57.874 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:35:57.817+0000 1694c700 -1 osd.3 332 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80878 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e330) 2019-11-29T17:37:52.918 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:52.903+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:53.601 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:53.594+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:53.858 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:53.852+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:53.920 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:53.913+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:53.961 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:53.955+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:54.604 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:54.594+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:54.633 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:54.624+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:54.737 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:54.726+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:54.836 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:54.828+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:54.952 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:54.945+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:54.955 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:54.947+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:55.596 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:55.588+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:55.616 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:55.608+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:55.739 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:55.733+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:55.833 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:55.826+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:55.913 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:55.904+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:55.964 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:55.958+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:55.987 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:37:55.971+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:37:56.643 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:56.634+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:56.664 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:56.656+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:56.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:56.750+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:56.850 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:56.844+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:56.940 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:56.931+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:57.007 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:57.001+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:57.016 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:37:57.008+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:37:57.643 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:57.634+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:57.650 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:57.642+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:57.788 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:57.781+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:57.833 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:57.826+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:57.944 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:57.935+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:58.013 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:58.004+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:58.037 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:37:58.028+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:37:58.675 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:58.667+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:58.692 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:58.683+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:58.818 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:58.811+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:58.834 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:58.824+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:58.907 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:58.897+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:37:59.013 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:37:59.007+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:37:59.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:37:59.013+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:37:59.680 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:37:59.672+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:37:59.725 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:37:59.716+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:37:59.843 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:37:59.836+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:37:59.864 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:37:59.857+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:37:59.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:37:59.859+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:00.051 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:00.043+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:00.067 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:00.061+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:00.720 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:00.712+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:00.727 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:00.719+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:00.808 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:00.799+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:00.844 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:00.837+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:00.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:00.890+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:01.086 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:01.074+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:01.098 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:01.091+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:01.743 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:01.733+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:01.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:01.776+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:01.816 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:01.796+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:01.887 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:01.878+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:01.901 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:01.890+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:02.096 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:02.089+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:02.130 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:02.123+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:02.796 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:02.788+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:02.798 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:02.790+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:02.841 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:02.828+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:02.887 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:02.881+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:02.943 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:02.934+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:03.077 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:03.069+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:03.114 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:03.106+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:03.771 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:03.763+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:03.838 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:03.829+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:03.871 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:03.864+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:03.874 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:03.867+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:03.963 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:03.955+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:04.044 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:04.036+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:04.115 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:04.109+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:04.795 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:04.786+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:04.818 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:04.811+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:04.843 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:04.837+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:04.845 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:04.839+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:04.981 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:04.964+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:05.056 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:05.047+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:05.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:05.110+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:05.835 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:05.827+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:05.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:05.833+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:05.889 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:05.883+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:05.893 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:05.884+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:05.968 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:05.962+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:06.083 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:06.076+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:06.101 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:06.088+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:06.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:06.801+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:06.817 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:06.807+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:06.876 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:06.870+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:06.891 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:06.884+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:06.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:06.923+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:07.099 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:07.089+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:07.153 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:07.141+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:07.807 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:07.800+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:07.850 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:07.843+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:07.885 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:07.868+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:07.885 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:07.875+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:07.958 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:07.949+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:08.080 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:08.073+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:08.126 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:08.118+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:08.810 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:08.803+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:08.857 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:08.851+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:08.870 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:08.863+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:08.907 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:08.901+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:09.010 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:09.003+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:09.085 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:09.078+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:09.134 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:09.127+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:09.770 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:09.762+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:09.910 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:09.902+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:09.910 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:09.903+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:09.924 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:09.917+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:10.047 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:10.040+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:10.127 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:10.119+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:10.188 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:10.172+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:10.727 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:10.719+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:10.888 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:10.881+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:10.952 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:10.944+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:10.963 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:10.955+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:11.030 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:11.018+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:11.153 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:11.144+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:11.193 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:11.186+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:11.719 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:11.711+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:11.851 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:11.844+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:11.917 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:11.910+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:11.944 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:11.933+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:12.062 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:12.054+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:12.112 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:12.105+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:12.246 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:12.238+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:12.769 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:12.759+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:12.854 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:12.847+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:12.929 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:12.923+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:12.936 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:12.930+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:13.018 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:13.011+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:13.087 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:13.080+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:13.251 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:13.243+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:13.745 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:13.737+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:13.888 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:13.880+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:13.920 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:13.913+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:13.921 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:13.914+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:14.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:14.031+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:14.103 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:14.094+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:14.297 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:14.289+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:14.755 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:14.746+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:14.907 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:14.899+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:14.910 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:14.903+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:14.922 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:14.908+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:15.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:15.043+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:15.139 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:15.133+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:15.319 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:15.311+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:15.773 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:15.764+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:15.870 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:15.859+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:15.963 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:15.955+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:15.965 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:15.958+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:16.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:16.048+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:16.183 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:16.176+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:16.353 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:16.337+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:16.733 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:16.721+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:16.824 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:16.818+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:16.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:16.936+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:16.998 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:16.991+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:17.059 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:17.051+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:17.216 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:17.208+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:17.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:17.306+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:17.763 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:17.755+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:17.872 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:17.865+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:17.974 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:17.963+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:17.997 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:17.989+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:18.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:18.018+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:18.252 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:18.245+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:18.340 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:18.335+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:18.750 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:18.742+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:18.898 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:18.890+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:18.965 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:18.958+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:18.981 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:18.976+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:18.989 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:18.982+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:19.240 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:19.232+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:19.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:19.372+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:19.763 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:19.755+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:19.897 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:19.892+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:19.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:19.942+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:19.958 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:19.951+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:19.976 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:19.970+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:20.241 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:20.226+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:20.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:20.353+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:20.739 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:20.731+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:20.883 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:20.877+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:20.952 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:20.947+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:20.955 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:20.948+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:20.996 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:20.989+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:21.279 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:21.271+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:21.364 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:21.350+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:21.768 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:21.752+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:21.878 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:21.871+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:21.918 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:21.910+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:21.978 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:21.970+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:21.982 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:21.975+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:22.329 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:22.323+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:22.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:22.393+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:22.729 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:22.721+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:22.896 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:22.890+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:22.966 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:22.958+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:22.979 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:22.973+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:23.013 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:23.003+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:23.321 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:23.314+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:23.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:23.377+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:23.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:23.699+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:23.935 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:23.928+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:23.988 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:23.980+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:24.022 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:24.016+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:24.026 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:24.018+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:24.289 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:24.282+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:24.345 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:24.338+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:24.690 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:24.683+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:24.950 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:24.943+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:24.954 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:24.948+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:25.007 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:25.001+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:25.037 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:25.029+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:25.260 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:25.253+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:25.330 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:25.322+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:25.707 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:25.700+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:25.910 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:38:25.903+0000 1694c700 -1 osd.1 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.894054+0000 front 2019-11-29T17:37:10.842625+0000 (oldest deadline 2019-11-29T17:37:53.729639+0000) 2019-11-29T17:38:25.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:38:25.912+0000 1694c700 -1 osd.7 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.412514+0000 front 2019-11-29T17:37:09.452324+0000 (oldest deadline 2019-11-29T17:37:53.496581+0000) 2019-11-29T17:38:26.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:38:26.030+0000 1694c700 -1 osd.3 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:10.600009+0000 front 2019-11-29T17:37:10.640676+0000 (oldest deadline 2019-11-29T17:37:54.061836+0000) 2019-11-29T17:38:26.057 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:38:26.050+0000 1694c700 -1 osd.6 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:11.173813+0000 front 2019-11-29T17:37:11.175451+0000 (oldest deadline 2019-11-29T17:37:52.744658+0000) 2019-11-29T17:38:26.308 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:38:26.300+0000 1694c700 -1 osd.4 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.678285+0000 front 2019-11-29T17:37:09.700491+0000 (oldest deadline 2019-11-29T17:37:55.453941+0000) 2019-11-29T17:38:26.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:38:26.350+0000 1694c700 -1 osd.2 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:08.112753+0000 front 2019-11-29T17:37:08.434199+0000 (oldest deadline 2019-11-29T17:37:52.106014+0000) 2019-11-29T17:38:26.429 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:38:26.419+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e336 v336) 2019-11-29T17:38:26.678 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:38:26.671+0000 1694c700 -1 osd.5 336 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:37:09.831805+0000 front 2019-11-29T17:37:09.915376+0000 (oldest deadline 2019-11-29T17:37:53.681314+0000) 2019-11-29T17:38:46.615 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:25:59.503 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.631 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:25:59.518 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.657 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:25:59.544 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.671 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:25:59.557 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.695 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:25:59.467 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.697 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:25:59.471 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.846 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:25:59.734 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:46.912 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:25:59.799 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:38:47.031 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:25:59.804 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.867 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:19.750 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.873 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:19.760 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.880 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:19.768 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.896 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:19.772 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.901 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:19.672 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.928 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:19.701 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.959 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:19.847 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:06.973 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:19.849 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:07.060 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:19.833 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:16.994 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:29.882 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.006 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:29.894 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.025 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:26:29.913 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.032 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:29.919 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.057 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:29.945 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.087 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:26:29.968 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.112 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:29.882 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.124 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:29.897 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:17.161 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:26:29.934 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:39:24.974 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:24.967+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:25.497 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:25.488+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:25.991 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:25.984+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:26.550 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:26.539+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:26.866 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:26.858+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:27.045 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:27.035+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:27.392 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:27.386+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:27.468 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:27.460+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:27.527 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:27.511+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:27.588 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:27.580+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:27.885 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:27.878+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:28.080 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:28.073+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:28.443 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:28.437+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:28.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:28.464+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:28.505 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:28.498+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:28.558 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:28.550+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:28.868 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:28.857+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:29.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:29.050+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:29.407 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:29.401+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:29.443 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:29.435+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:29.532 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:29.524+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:29.581 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:29.573+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:29.791 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:29.784+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:29.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:29.879+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:30.029 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:30.021+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:30.413 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:30.399+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:30.430 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:30.423+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:30.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:30.543+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:30.597 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:30.588+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:30.797 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:30.789+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:30.855 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:30.849+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:31.056 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:31.049+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:31.411 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:31.403+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:31.456 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:31.449+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:31.580 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:31.574+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:31.612 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:31.604+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:31.808 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:31.802+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:31.901 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:31.894+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:32.035 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:32.027+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:32.434 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:32.428+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:32.472 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:32.466+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:32.589 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:32.582+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:32.604 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:32.597+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:32.860 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:32.854+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:32.875 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:32.868+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:33.089 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:33.074+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:33.429 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:33.423+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:33.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:33.493+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:33.614 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:33.607+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:33.630 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:33.623+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:33.884 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:33.877+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:33.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:33.890+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:34.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:34.064+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:34.431 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:34.424+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:34.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:34.491+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:34.619 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:34.613+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:34.634 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:34.628+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:34.872 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:34.858+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:34.929 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:34.923+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:35.036 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:35.030+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:35.463 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:35.456+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:35.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:35.463+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:35.585 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:35.578+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:35.603 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:35.596+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:35.910 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:35.903+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:35.948 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:35.942+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:36.004 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:35.997+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:36.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:36.422+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:36.439 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:36.432+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:36.586 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:36.579+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:36.644 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:36.638+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:36.919 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:36.912+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:36.961 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:36.954+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:36.961 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:36.955+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:37.016 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:37.009+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:37.446 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:37.440+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:37.448 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:37.442+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:37.562 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:37.554+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:37.697 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:37.690+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:37.964 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:37.955+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:37.966 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:37.959+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:37.973 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:37.966+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:37.995 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:37.987+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:38.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:38.428+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:38.448 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:38.441+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:38.583 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:38.577+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:38.711 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:38.704+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:38.924 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:38.911+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:38.927 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:38.920+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:39.009 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:38.998+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:39.018 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:39.010+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:39.408 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:39.401+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:39.472 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:39.465+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:39.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:39.580+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:39.691 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:39.683+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:39.901 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:39.893+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:39.902 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:39.896+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:39.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:39.992+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:40.019 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:40.013+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:40.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:40.377+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:40.495 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:40.488+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:40.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:40.541+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:40.742 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:40.735+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:40.865 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:40.857+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:40.866 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:40.859+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:40.957 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:40.950+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:41.003 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:40.996+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:41.406 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:41.400+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:41.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:41.540+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:41.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:41.580+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:41.760 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:41.752+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:41.905 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:41.898+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:41.906 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:41.900+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:41.948 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:41.941+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:42.044 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:42.037+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:42.420 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:42.413+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:42.540 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:42.533+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:42.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:42.580+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:42.726 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:42.719+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:42.950 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:42.941+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:42.951 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:42.945+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:43.005 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:42.992+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:43.033 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:43.025+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:43.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:43.378+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:43.530 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:43.523+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:43.639 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:43.632+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:43.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:43.740+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:43.985 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:43.978+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:43.989 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:43.982+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:44.017 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:44.010+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:44.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:44.031+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:44.354 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:44.347+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:44.561 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:44.554+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:44.622 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:44.611+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:44.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:44.764+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:45.002 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:44.994+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:45.007 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:45.000+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:45.010 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:45.004+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:45.012 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:45.006+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:45.331 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:45.323+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:45.522 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:45.514+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:45.611 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:45.604+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:45.805 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:45.797+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:46.006 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:45.997+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:46.024 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:46.016+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:46.028 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:46.019+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:46.054 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:46.046+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:46.290 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:46.283+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:46.545 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:46.538+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:46.637 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:46.630+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:46.783 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:46.776+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:46.982 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:46.974+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:47.017 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:47.009+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:47.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:47.035+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:47.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:47.036+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:47.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:47.307+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:47.574 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:47.557+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:47.633 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:47.625+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:47.760 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:47.753+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:47.989 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:47.982+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:48.020 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:48.010+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:48.074 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:48.067+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:48.075 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:48.069+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:48.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:48.282+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:48.599 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:48.591+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:48.666 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:48.660+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:48.740 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:48.731+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:48.977 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:48.970+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:48.987 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:48.981+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:49.111 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:49.106+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:49.114 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:49.109+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:49.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:49.280+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:49.621 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:49.612+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:49.699 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:49.690+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:49.700 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:49.694+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:49.952 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:49.944+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:49.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:49.991+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:50.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:50.112+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:50.122 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:50.117+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:50.293 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:50.286+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:50.606 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:50.597+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:50.693 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:50.683+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:50.753 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:50.745+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:50.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:50.912+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:51.021 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:51.013+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:51.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:51.162+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:51.168 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:51.164+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:51.268 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:51.261+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:51.624 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:51.616+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:51.682 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:51.674+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:51.797 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:51.791+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:51.908 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:51.900+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:52.074 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:52.066+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:52.181 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:52.175+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:52.181 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:52.177+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:52.231 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:52.225+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:52.410 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:39:52.399+0000 20e22700 -1 RGWWatcher::handle_error cookie 5485669168 err (107) Transport endpoint is not connected 2019-11-29T17:39:52.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:52.588+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:52.727 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:52.719+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:52.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:52.762+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:52.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:52.890+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:53.088 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:53.077+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:53.181 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:53.176+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:53.182 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:53.178+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:53.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:53.183+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:53.566 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:53.558+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:53.710 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:53.702+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:53.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:53.756+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:53.922 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:53.914+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:54.125 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:54.117+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:54.168 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:54.162+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:54.170 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:54.165+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:54.181 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:54.175+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:54.611 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:54.603+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:54.734 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:54.723+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:54.814 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:54.808+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:54.953 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:54.944+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:55.134 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:55.125+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:55.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:55.187+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:55.211 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:55.204+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:55.212 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:55.207+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:55.627 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:55.617+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:55.732 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:55.724+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:55.832 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:55.826+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:55.928 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:55.919+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:56.156 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:56.147+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:56.190 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:56.182+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:56.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:56.185+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:56.203 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:56.198+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:56.626 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:56.619+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:56.752 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:56.745+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:56.870 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:56.863+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:56.884 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:56.876+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:57.176 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:57.167+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:57.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:57.202+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:57.213 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:57.204+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:39:57.216 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:57.210+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:57.217 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:57.213+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:57.628 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:57.621+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:57.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:57.771+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:57.864 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:57.856+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:57.874 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:57.868+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:58.138 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:58.130+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:58.204 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:58.199+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:58.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:58.201+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:39:58.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:58.225+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:58.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:58.226+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:58.648 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:58.641+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:58.828 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:58.819+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:58.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:58.862+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:58.900 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:58.895+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:39:59.189 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:39:59.182+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:39:59.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:59.195+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:39:59.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:39:59.197+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:39:59.222 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:59.213+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:39:59.222 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:39:59.216+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:39:59.641 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:39:59.634+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:39:59.782 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:39:59.775+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:39:59.909 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:39:59.897+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:39:59.928 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:39:59.916+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:00.148 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:00.141+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:00.214 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:00.207+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:00.216 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:00.210+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:00.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:00.241+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:00.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:00.244+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:00.642 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:00.635+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:00.785 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:00.777+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:00.946 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:00.940+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:00.952 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:00.945+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:01.156 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:01.146+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:01.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:01.217+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:01.223 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:01.219+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:01.233 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:01.227+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:01.234 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:01.229+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:01.633 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:01.626+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:01.756 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:01.749+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:01.917 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:01.910+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:01.973 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:01.966+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:02.082 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:40:02.074+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 48sec e341 v341) 2019-11-29T17:40:02.186 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:02.180+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:02.259 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:02.252+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:02.261 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:02.256+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:02.267 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:02.260+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:02.268 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:02.263+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:02.686 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:02.678+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:02.718 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:02.710+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:02.951 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:02.945+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:03.009 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:03.002+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:03.215 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:03.207+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:03.257 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:03.250+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:03.258 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:03.253+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:03.263 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:03.257+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:03.264 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:03.259+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:03.677 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:03.670+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:03.692 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:03.685+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:03.922 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:03.916+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:04.062 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:04.055+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:04.267 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:04.259+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:04.280 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:04.275+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:04.284 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:04.278+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:04.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:04.286+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:04.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:04.287+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:04.663 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:04.656+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:04.716 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:04.710+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:04.900 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:04.894+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:05.110 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:05.098+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:05.252 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:05.242+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:05.271 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:05.265+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:05.271 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:05.266+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:05.324 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:05.316+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:05.326 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:05.320+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:05.667 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:05.660+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:05.741 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:05.733+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:05.862 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:05.856+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:06.078 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:06.070+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:06.260 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:06.252+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:06.286 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:06.280+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:06.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:06.283+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:06.354 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:06.346+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:06.355 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:06.349+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:06.641 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:06.633+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:06.713 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:06.707+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:06.860 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:06.854+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:07.087 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:40:07.083+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 48sec e341 v341) 2019-11-29T17:40:07.109 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:07.102+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:07.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:07.289+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:07.297 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:07.292+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:07.318 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:07.305+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:07.354 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:07.347+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:07.355 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:07.350+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:07.607 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:07.600+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:07.739 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:07.733+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:07.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:07.895+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:08.105 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:08.099+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:08.279 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:08.272+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:08.301 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:08.291+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:08.301 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:08.293+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:08.335 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:08.326+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:08.345 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:08.331+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:08.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:08.643+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:08.784 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:08.777+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:08.859 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:08.853+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:09.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:09.140+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:09.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:09.263+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:09.269 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:09.265+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:09.301 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:09.295+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:09.303 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:09.299+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:09.311 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:09.302+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:09.698 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:09.690+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:09.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:09.802+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:09.822 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:09.815+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:10.105 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:10.098+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:10.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:10.283+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:10.292 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:10.287+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:10.334 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:10.327+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:10.338 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:10.333+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:10.370 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:10.357+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:10.681 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:10.663+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:10.828 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:10.822+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:10.831 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:10.824+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:11.126 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:11.118+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:11.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:11.260+0000 1694c700 -1 osd.3 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:43.778601+0000 front 2019-11-29T17:38:43.769129+0000 (oldest deadline 2019-11-29T17:39:26.489436+0000) 2019-11-29T17:40:11.268 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:11.263+0000 1694c700 -1 osd.3 341 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:11.379 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:40:11.367+0000 1694c700 -1 osd.4 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:41.273872+0000 front 2019-11-29T17:38:41.226301+0000 (oldest deadline 2019-11-29T17:39:26.124094+0000) 2019-11-29T17:40:11.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:11.376+0000 1694c700 -1 osd.0 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:38:56.652809+0000 (oldest deadline 2019-11-29T17:39:36.652809+0000) 2019-11-29T17:40:11.401 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:40:11.380+0000 1694c700 -1 osd.0 341 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e340) 2019-11-29T17:40:11.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:40:11.695+0000 1694c700 -1 osd.5 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.361768+0000 front 2019-11-29T17:38:42.576221+0000 (oldest deadline 2019-11-29T17:39:27.441268+0000) 2019-11-29T17:40:11.801 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:40:11.791+0000 1694c700 -1 osd.6 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:39.339262+0000 front 2019-11-29T17:38:39.336790+0000 (oldest deadline 2019-11-29T17:39:24.572173+0000) 2019-11-29T17:40:11.814 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:40:11.808+0000 1694c700 -1 osd.2 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:42.046609+0000 front 2019-11-29T17:38:42.047392+0000 (oldest deadline 2019-11-29T17:39:27.328890+0000) 2019-11-29T17:40:12.097 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:40:12.090+0000 1694c700 -1 osd.7 341 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:38:38.831506+0000 front 2019-11-29T17:38:38.832242+0000 (oldest deadline 2019-11-29T17:39:24.458484+0000) 2019-11-29T17:40:12.144 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:40:12.130+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 48sec e341 v341) 2019-11-29T17:40:12.321 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:12.314+0000 1694c700 -1 osd.3 342 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:13.341 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:40:13.335+0000 1694c700 -1 osd.3 342 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:105848 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485669168 gen 1] snapc 0=[] ondisk+write+known_if_redirected e341) 2019-11-29T17:40:34.177 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:40:33.952+0000 1694c700 -1 osd.1 341 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:105729 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 5] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-11-29T17:41:08.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:41:08.133+0000 1694c700 -1 osd.0 346 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=66 ondisk+retry+read+known_if_redirected e344) 2019-11-29T17:41:09.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:41:09.142+0000 1694c700 -1 osd.0 346 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=66 ondisk+retry+read+known_if_redirected e344) 2019-11-29T17:41:10.277 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:41:10.265+0000 1694c700 -1 osd.0 346 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=66 ondisk+retry+read+known_if_redirected e344) 2019-11-29T17:42:01.425 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:01.417+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:01.829 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:01.813+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:02.386 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:02.379+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:02.464 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:02.457+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:02.798 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:02.778+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:03.229 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:03.221+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:03.436 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:03.430+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:03.486 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:03.476+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:03.680 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:03.673+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:03.756 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:03.749+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:04.179 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:04.171+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:04.273 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:04.264+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:04.442 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:04.436+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:04.459 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:04.453+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:04.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:04.660+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:04.799 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:04.792+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:05.140 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:05.133+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:05.204 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:05.196+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:05.259 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:05.251+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:05.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:05.413+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:05.474 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:05.466+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:05.664 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:05.656+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:05.831 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:05.825+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:06.180 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:06.173+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:06.181 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:06.173+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:06.280 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:06.272+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:06.465 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:06.457+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:06.476 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:06.470+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:06.716 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:06.708+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:06.833 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:06.826+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:07.216 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:07.208+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:07.239 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:07.228+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:07.254 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:07.246+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:07.494 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:07.487+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:07.520 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:07.502+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:07.707 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:07.700+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:07.875 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:07.861+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:08.210 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:08.201+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:08.229 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:08.221+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:08.250 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:08.243+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:08.513 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:08.506+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:08.520 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:08.513+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:08.679 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:08.672+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:08.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:08.907+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:09.210 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:09.202+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:09.219 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:09.211+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:09.235 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:09.228+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:09.485 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:09.480+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:09.528 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:09.522+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:09.635 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:09.627+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:09.959 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:09.954+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:10.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:10.180+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:10.196 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:10.189+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:10.211 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:10.203+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:10.460 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:10.453+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:10.501 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:10.490+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:10.624 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:10.616+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:10.971 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:10.965+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:11.191 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:11.183+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:11.231 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:11.224+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:11.243 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:11.236+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:11.454 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:11.447+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:11.503 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:11.498+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:11.628 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:11.620+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:11.980 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:11.973+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:12.204 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:12.191+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:12.261 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:12.253+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:12.264 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:12.257+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:12.493 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:12.487+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:12.511 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:12.505+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:12.651 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:12.643+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:12.966 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:12.961+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:13.226 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:13.218+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:13.230 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:13.222+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:13.294 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:13.287+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:13.504 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:13.498+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:13.528 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:13.521+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:13.681 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:13.675+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:14.000 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:13.995+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:14.215 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:14.206+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:14.256 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:14.249+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:14.279 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:14.271+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:14.512 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:14.505+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:14.572 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:14.566+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:14.681 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:14.673+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:14.992 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:14.985+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:15.224 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:15.217+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:15.254 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:15.247+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:15.285 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:15.278+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:15.556 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:15.549+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:15.572 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:15.566+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:15.724 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:15.712+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:16.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:16.009+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:16.253 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:16.246+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:16.271 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:16.263+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:16.300 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:16.291+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:16.557 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:16.546+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:16.557 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:16.551+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:16.694 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:16.686+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:17.027 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:17.020+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:17.284 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:17.262+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:17.314 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:17.307+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:17.323 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:17.316+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:17.533 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:17.525+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:17.548 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:17.542+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:17.694 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:17.682+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:18.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:18.040+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:18.317 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:18.310+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:18.318 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:18.310+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:18.362 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:18.354+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:18.533 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:18.527+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:18.565 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:18.559+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:18.655 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:18.648+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:19.036 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:19.029+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:19.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:19.274+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:19.319 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:19.311+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:19.361 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:19.349+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:19.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:19.582+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:19.614 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:19.608+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:19.691 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:19.684+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:20.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:20.039+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:20.279 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:20.272+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:20.318 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:20.310+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:20.330 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:20.321+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:20.613 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:20.605+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:20.637 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:20.630+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:20.735 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:20.726+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:21.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:21.005+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:21.267 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:21.258+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:21.286 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:21.277+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:21.289 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:21.281+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:21.595 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:21.588+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:21.632 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:21.626+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:21.739 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:21.730+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:22.050 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:22.043+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:22.247 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:42:22.226+0000 1694c700 -1 osd.4 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.383865+0000 front 2019-11-29T17:41:20.931077+0000 (oldest deadline 2019-11-29T17:42:02.798897+0000) 2019-11-29T17:42:22.248 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:22.240+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:22.297 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:22.289+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:22.605 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:22.599+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:22.643 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:22.638+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:22.783 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:22.773+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:23.081 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:23.075+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:23.298 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:23.289+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:23.299 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:23.290+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:23.585 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:23.578+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:23.643 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:23.636+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:23.832 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:23.824+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:24.050 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:24.044+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:24.292 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:24.280+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:24.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:24.299+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:24.619 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:24.607+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:24.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:24.681+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:24.810 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:24.801+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:25.068 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:25.062+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:25.283 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:25.274+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:25.338 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:25.330+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:25.661 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:25.655+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:25.744 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:25.727+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:25.862 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:25.852+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:26.043 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:26.036+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:26.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:26.273+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:26.367 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:26.358+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:26.635 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:26.628+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:26.751 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:26.744+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:26.823 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:26.815+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:26.997 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:26.992+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:27.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:27.245+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:27.402 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:27.392+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:27.637 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:27.630+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:27.788 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:27.781+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:27.814 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:27.804+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:28.005 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:27.997+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:28.295 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:28.286+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:28.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:28.410+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:28.618 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:28.610+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:28.782 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:28.774+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:28.788 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:28.782+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:29.030 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:29.023+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:29.339 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:29.331+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:29.397 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:29.389+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:29.625 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:29.620+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:29.785 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:29.778+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:29.816 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:29.802+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:30.011 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:30.005+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:30.346 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:30.338+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:30.422 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:30.414+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:30.668 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:30.656+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:30.745 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:30.738+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:30.802 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:30.795+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:30.974 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:30.968+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:31.388 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:31.381+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:31.397 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:31.391+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:31.648 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:31.641+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:31.711 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:31.703+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:31.777 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:31.769+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:31.969 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:31.962+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:32.376 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:32.370+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:32.410 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:32.403+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:32.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:32.661+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:32.685 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:32.678+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:32.793 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:32.786+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:32.947 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:32.939+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:33.400 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:33.392+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:33.412 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:33.406+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:33.644 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:33.637+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:33.708 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:33.701+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:33.778 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:33.770+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:33.957 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:33.950+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:34.362 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:34.355+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:34.446 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:34.437+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:34.658 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:34.651+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:34.669 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:34.661+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:34.788 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:34.781+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:34.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:34.970+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:35.375 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:35.368+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:35.446 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:35.439+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:35.663 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:35.655+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:35.717 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:35.709+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:35.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:35.739+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:35.997 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:35.991+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:36.402 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:36.395+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:36.421 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:36.414+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:36.710 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:36.703+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:36.716 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:36.710+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:36.758 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:36.751+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:42:37.026 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:42:37.019+0000 1694c700 -1 osd.3 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.928735+0000 front 2019-11-29T17:41:18.111885+0000 (oldest deadline 2019-11-29T17:42:01.378172+0000) 2019-11-29T17:42:37.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:42:37.417+0000 1694c700 -1 osd.5 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:19.356825+0000 front 2019-11-29T17:41:19.322835+0000 (oldest deadline 2019-11-29T17:42:05.124651+0000) 2019-11-29T17:42:37.449 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:42:37.441+0000 1694c700 -1 osd.7 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:17.927871+0000 front 2019-11-29T17:41:17.791650+0000 (oldest deadline 2019-11-29T17:42:03.419431+0000) 2019-11-29T17:42:37.659 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:42:37.634+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e346 v346) 2019-11-29T17:42:37.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:42:37.677+0000 1694c700 -1 osd.2 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:14.626050+0000 front 2019-11-29T17:41:18.854713+0000 (oldest deadline 2019-11-29T17:42:02.212564+0000) 2019-11-29T17:42:37.709 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:42:37.700+0000 1694c700 -1 osd.1 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:15.828967+0000 front 2019-11-29T17:41:16.323762+0000 (oldest deadline 2019-11-29T17:42:01.119890+0000) 2019-11-29T17:42:37.733 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:42:37.725+0000 1694c700 -1 osd.6 346 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:41:20.070539+0000 front 2019-11-29T17:41:20.382571+0000 (oldest deadline 2019-11-29T17:42:03.427621+0000) 2019-11-29T17:43:10.743 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:30:23.631 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:10.755 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:30:23.638 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:10.761 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:30:23.534 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:10.768 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:30:23.541 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:10.775 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:30:23.662 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:10.781 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:30:23.668 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:11.031 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:30:23.804 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:11.053 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:30:23.940 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:11.085 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:30:23.966 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:43:37.097 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:43:36.988+0000 20e22700 -1 RGWWatcher::handle_error cookie 5464921824 err (107) Transport endpoint is not connected 2019-11-29T17:43:40.599 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:40.593+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:41.587 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:41.581+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:42.547 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:42.541+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:43.511 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:43.506+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:44.546 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:44.540+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:45.546 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:45.541+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:46.596 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:46.591+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:47.595 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:47.589+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:48.627 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:48.621+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:49.613 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:49.607+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:50.598 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:50.587+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:51.590 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:51.585+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:52.561 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:52.556+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:53.594 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:53.590+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:54.619 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:54.613+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:55.610 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:55.604+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:56.635 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:56.629+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:57.679 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:57.672+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:58.660 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:58.654+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:43:59.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:43:59.667+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:00.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:00.630+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:01.629 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:01.623+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:02.650 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:02.640+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:03.501 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:44:03.484+0000 20e22700 -1 RGWWatcher::handle_error cookie 440731632 err (107) Transport endpoint is not connected 2019-11-29T17:44:03.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:03.681+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:04.725 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:04.719+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:05.742 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:05.735+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:06.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:06.733+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:07.699 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:07.692+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:08.096 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:08.091+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:08.663 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:08.654+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:09.133 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:09.127+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:09.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:09.648+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:10.169 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:10.163+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:10.703 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:10.696+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:11.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:11.196+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:11.735 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:11.729+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:12.158 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:12.152+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:12.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:12.742+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:13.197 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:13.191+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:13.722 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:13.716+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:14.248 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:14.242+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:14.745 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:14.738+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:15.210 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:15.204+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:15.702 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:15.696+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:16.215 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:16.208+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:16.747 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:16.739+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:17.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:17.188+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:17.790 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:17.783+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:18.231 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:18.224+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:18.782 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:18.775+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:19.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:19.177+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:19.802 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:19.796+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:19.988 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:19.979+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:20.163 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:20.157+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:20.479 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:20.471+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:20.818 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:20.810+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:20.830 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:20.825+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:21.006 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:20.998+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:21.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:21.180+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:21.520 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:21.507+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:21.796 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:21.790+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:21.802 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:21.796+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:22.014 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:22.006+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:22.179 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:22.169+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:22.230 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:22.225+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:22.499 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:22.491+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:22.814 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:22.808+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:22.823 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:22.817+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:23.040 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:23.031+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:23.134 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:23.127+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:23.200 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:23.194+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:23.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:23.195+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:23.519 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:23.510+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:23.846 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:23.840+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:23.849 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:23.843+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:23.856 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:23.850+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:24.059 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:24.052+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:24.112 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:24.104+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:24.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:24.231+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:24.241 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:24.236+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:24.297 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:44:24.285+0000 1694c700 -1 osd.6 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.907740+0000 front 2019-11-29T17:43:39.789947+0000 (oldest deadline 2019-11-29T17:44:23.578162+0000) 2019-11-29T17:44:24.509 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:24.501+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:24.832 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:24.822+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:24.833 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:24.827+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:24.870 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:24.863+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:25.098 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:25.091+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:25.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:25.134+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:25.248 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:25.242+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:25.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:25.243+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:25.312 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:44:25.305+0000 1694c700 -1 osd.6 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.907740+0000 front 2019-11-29T17:43:39.789947+0000 (oldest deadline 2019-11-29T17:44:23.578162+0000) 2019-11-29T17:44:25.520 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:25.513+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:25.806 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:25.799+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:25.807 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:25.801+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:25.877 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:25.869+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:26.122 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:26.104+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:26.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:26.131+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:26.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:26.197+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:26.206 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:26.201+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:26.326 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:44:26.319+0000 1694c700 -1 osd.6 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.907740+0000 front 2019-11-29T17:43:39.789947+0000 (oldest deadline 2019-11-29T17:44:23.578162+0000) 2019-11-29T17:44:26.539 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:26.531+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:26.843 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:26.836+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:26.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:26.838+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:26.893 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:26.887+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:27.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:27.092+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:27.153 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:27.146+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:27.200 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:27.168+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:27.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:27.171+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:27.530 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:27.519+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:27.843 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:27.835+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:27.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:27.838+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:27.938 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:27.932+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:28.096 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:28.089+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:28.134 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:28.125+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:28.136 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:28.128+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:28.148 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:28.141+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:28.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:28.474+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:28.805 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:28.799+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:28.806 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:28.800+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:28.927 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:28.921+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:29.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:29.081+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:29.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:29.083+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:29.093 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:29.086+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:29.113 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:29.105+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:29.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:29.444+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:29.786 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:29.778+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:29.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:29.781+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:29.976 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:29.970+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:30.062 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:30.056+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:30.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:30.059+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:30.067 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:30.061+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:30.099 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:30.092+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:30.496 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:30.489+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:30.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:30.774+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:30.787 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:30.779+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:31.016 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:31.010+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:31.081 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:31.073+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:31.087 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:31.080+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:31.108 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:31.102+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:31.109 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:31.103+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:31.486 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:31.478+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:31.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:31.778+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:31.786 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:31.781+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:31.998 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:31.991+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:32.081 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:32.074+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:32.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:32.081+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:32.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:32.082+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:32.111 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:32.098+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:32.447 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:32.440+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:32.766 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:32.755+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:32.767 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:32.759+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:32.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:32.971+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:33.061 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:33.054+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:33.062 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:33.056+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:33.120 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:33.097+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:33.120 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:33.100+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:33.500 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:33.492+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:33.730 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:33.725+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:33.732 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:33.726+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:33.951 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:33.946+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:34.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:34.029+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:34.038 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:34.032+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:34.111 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:34.103+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:34.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:34.109+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:34.506 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:34.498+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:34.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:34.756+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:34.770 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:34.759+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:34.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:34.980+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:35.043 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:35.036+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:35.044 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:35.039+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:35.104 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:35.092+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:35.116 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:35.108+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:35.484 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:35.475+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:35.789 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:35.780+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:35.791 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:35.784+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:35.954 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:35.948+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:36.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:36.040+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:36.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:36.042+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:36.111 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:36.104+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:36.154 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:36.146+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:36.474 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:36.467+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:36.808 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:36.801+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:36.810 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:36.803+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:36.909 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:36.903+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:37.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:37.001+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:37.010 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:37.004+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:37.105 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:37.098+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:37.168 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:37.157+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:37.477 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:37.470+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:37.863 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:37.854+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:37.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:37.858+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:37.933 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:37.927+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:37.999 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:37.993+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:38.002 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:37.997+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:38.074 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:38.065+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:38.182 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:38.174+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:38.530 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:38.522+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:38.873 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:38.861+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:38.873 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:38.863+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:38.966 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:38.960+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:38.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:38.962+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:38.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:38.978+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:39.043 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:39.036+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:39.140 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:39.133+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:39.484 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:39.477+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:39.912 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:39.905+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:39.918 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:39.912+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:39.944 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:39.938+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:39.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:39.969+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:39.978 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:39.973+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:40.006 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:39.998+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:40.149 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:40.141+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:40.516 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:40.509+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:40.876 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:40.868+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:40.876 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:40.871+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:40.972 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:40.965+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:40.973 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:40.967+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:40.983 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:40.977+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:41.004 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:40.996+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:41.148 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:41.140+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:41.550 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:41.543+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:41.924 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:41.914+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:41.925 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:41.918+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:41.953 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:41.947+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:41.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:41.951+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:42.025 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:42.019+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:42.029 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:42.021+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:42.134 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:42.126+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:42.529 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:42.521+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:42.920 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:42.914+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:42.920 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:42.915+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:42.925 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:42.919+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:42.926 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:42.921+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:43.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:43.024+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:43.064 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:44:43.057+0000 1694c700 -1 osd.4 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:37.811641+0000 front 2019-11-29T17:43:37.809726+0000 (oldest deadline 2019-11-29T17:44:19.745215+0000) 2019-11-29T17:44:43.158 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:43.151+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:43.549 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:43.541+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:43.897 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:43.890+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:43.897 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:43.892+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:43.947 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:43.942+0000 1694c700 -1 osd.0 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.125693+0000 front 2019-11-29T17:43:38.128622+0000 (oldest deadline 2019-11-29T17:44:22.808134+0000) 2019-11-29T17:44:43.949 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:43.945+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:44.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:44.000+0000 1694c700 -1 osd.2 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:44.210 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:44.202+0000 1694c700 -1 osd.7 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:44.568 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:44.562+0000 1694c700 -1 osd.5 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:44.929 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:44.923+0000 1694c700 -1 osd.3 352 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:44.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:44.925+0000 1694c700 -1 osd.3 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:106289 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 440731632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e352) 2019-11-29T17:44:44.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:44:44.980+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.654781+0000 front 2019-11-29T17:43:38.729981+0000 (oldest deadline 2019-11-29T17:44:20.079358+0000) 2019-11-29T17:44:45.020 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:44.994+0000 1694c700 -1 osd.0 352 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:45.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:45.249+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:45.678 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:44:45.627+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.757162+0000 front 2019-11-29T17:43:38.044060+0000 (oldest deadline 2019-11-29T17:44:19.646788+0000) 2019-11-29T17:44:45.894 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:45.888+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:46.121 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:46.093+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e350) 2019-11-29T17:44:46.297 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:44:46.289+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:38.769724+0000 front 2019-11-29T17:43:38.865766+0000 (oldest deadline 2019-11-29T17:44:21.560837+0000) 2019-11-29T17:44:46.927 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:44:46.921+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:43:39.664546+0000 front 2019-11-29T17:43:39.523821+0000 (oldest deadline 2019-11-29T17:44:22.961145+0000) 2019-11-29T17:44:47.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:47.141+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:48.120 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:48.114+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:49.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:49.160+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:50.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:50.185+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:51.175 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:51.169+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:52.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:52.164+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:53.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:53.188+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:54.159 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:54.153+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:55.127 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:55.120+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:56.098 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:56.089+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:57.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:57.125+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:58.118 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:58.112+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:44:59.110 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:44:59.103+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:00.093 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:00.087+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:01.121 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:01.114+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:02.150 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:02.143+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:03.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:03.142+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:04.133 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:04.127+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:05.184 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:05.177+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:106245 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5464921824 gen 11] snapc 0=[] ondisk+write+known_if_redirected e350) 2019-11-29T17:45:06.230 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:06.224+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:07.224 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:07.219+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:08.263 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:08.259+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:09.272 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:09.267+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:10.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:10.293+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:11.300 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:11.295+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:12.344 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:12.339+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:13.357 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:13.352+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:14.373 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:14.368+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:15.766 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:15.684+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:16.946 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:16.940+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:18.237 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:18.180+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:45:19.463 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:45:19.257+0000 1694c700 -1 osd.0 353 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=68 ondisk+retry+read+known_if_redirected e350) 2019-11-29T17:46:10.708 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:10.699+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:10.825 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:10.816+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:11.470 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:11.463+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:11.689 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:11.680+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:11.817 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:11.809+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:11.932 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:11.926+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:12.432 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:12.425+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:12.704 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:12.695+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:12.868 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:12.860+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:12.959 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:12.953+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:13.216 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:13.206+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:13.260 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:13.250+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:13.400 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:13.395+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:13.670 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:13.661+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:13.757 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:13.749+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:13.845 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:13.837+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:14.007 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:14.002+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:14.223 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:14.215+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:14.233 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:14.226+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:14.368 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:14.361+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:14.722 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:14.714+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:14.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:14.724+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:14.843 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:14.832+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:15.000 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:14.993+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:15.205 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:15.197+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:15.287 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:15.280+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:15.374 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:15.368+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:15.690 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:15.681+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:15.759 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:15.750+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:15.846 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:15.838+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:16.049 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:16.042+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:16.213 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:16.204+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:16.311 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:16.302+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:16.402 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:16.394+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:16.693 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:16.683+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:16.744 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:16.737+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:16.822 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:16.814+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:17.100 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:17.094+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:17.269 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:17.257+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:17.302 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:17.296+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:17.375 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:17.369+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:17.657 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:17.647+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:17.792 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:17.784+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:17.833 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:17.825+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:18.071 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:18.064+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:18.237 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:18.230+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:18.313 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:18.305+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:18.424 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:18.416+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:18.699 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:18.691+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:18.833 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:18.824+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:18.875 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:18.867+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:19.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:19.033+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:19.267 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:19.259+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:19.356 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:19.349+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:19.453 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:19.445+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:19.743 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:19.735+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:19.827 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:19.820+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:19.851 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:19.844+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:20.032 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:20.023+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:20.251 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:20.243+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:20.321 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:20.314+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:20.415 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:20.407+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:20.709 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:20.699+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:20.818 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:20.810+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:20.874 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:20.867+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:21.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:21.033+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:21.266 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:21.258+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:21.324 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:21.316+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:21.426 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:21.418+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:21.701 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:21.693+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:21.793 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:21.786+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:21.859 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:21.851+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:22.081 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:22.072+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:22.293 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:22.263+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:22.330 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:22.322+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:22.452 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:22.446+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:22.722 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:22.714+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:22.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:22.756+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:22.842 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:22.835+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:23.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:23.035+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:23.293 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:23.285+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:23.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:23.301+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:23.424 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:23.418+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:23.680 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:23.671+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:23.773 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:23.765+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:23.856 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:23.848+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:24.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:24.051+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:24.316 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:24.308+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:24.325 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:24.317+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:24.392 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:24.386+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:24.703 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:24.695+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:24.773 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:24.765+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:24.832 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:24.825+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:25.033 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:25.024+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:25.273 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:25.265+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:25.287 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:25.279+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:25.388 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:25.382+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:25.676 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:25.666+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:25.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:25.756+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:25.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:25.799+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:26.053 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:26.045+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:26.283 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:26.276+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:26.302 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:26.295+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:26.422 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:26.414+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:26.717 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:26.709+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:26.785 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:26.777+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:26.845 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:26.836+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:27.080 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:27.072+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:27.245 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:27.238+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:27.305 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:27.297+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:27.451 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:27.444+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:27.750 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:27.742+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:27.813 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:27.805+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:27.884 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:27.876+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:28.125 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:28.117+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:28.278 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:28.270+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:28.296 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:28.289+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:28.500 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:28.494+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:28.720 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:28.712+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:28.867 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:28.860+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:28.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:28.860+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:29.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:29.128+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:29.233 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:29.225+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:29.341 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:29.328+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:29.533 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:29.526+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:29.730 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:29.722+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:29.865 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:29.857+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:29.920 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:29.912+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:30.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:30.178+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:30.217 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:30.208+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:30.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:30.378+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:30.543 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:30.535+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:30.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:30.715+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:30.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:30.848+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:30.953 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:30.947+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:31.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:31.204+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:31.265 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:31.257+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:31.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:31.341+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:31.506 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:31.498+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:31.717 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:31.705+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:31.832 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:31.824+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:31.970 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:31.962+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:32.214 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:32.203+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:32.302 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:32.295+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:32.351 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:32.344+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:32.514 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:32.505+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:32.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:32.746+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:32.824 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:32.816+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:32.968 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:32.960+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:33.252 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:33.245+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:33.304 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:33.297+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:33.337 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:33.330+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:33.471 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:33.463+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:33.742 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:33.735+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:33.858 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:33.850+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:34.010 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:34.002+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:34.244 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:34.238+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:34.294 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:34.282+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:34.372 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:34.365+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:34.484 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:34.472+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:34.779 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:34.771+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:34.887 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:34.878+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:34.977 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:34.969+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:35.231 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:35.225+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:35.335 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:35.329+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:35.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:35.348+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:35.508 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:35.501+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:35.811 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:35.803+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:35.870 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:35.863+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:36.016 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:36.008+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:36.269 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:36.263+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:36.351 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:36.343+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:36.372 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:36.365+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:36.553 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:36.546+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:36.777 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:36.766+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:36.844 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:36.836+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:36.980 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:36.973+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:37.293 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:37.287+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:37.402 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:37.379+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:37.402 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:37.378+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:37.507 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:37.502+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:37.787 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:37.779+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:37.886 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:37.878+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:37.989 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:37.981+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:38.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:38.299+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:38.345 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:38.338+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:38.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:38.343+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:38.542 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:38.535+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:38.809 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:38.796+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:38.852 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:38.845+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:38.956 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:38.942+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:39.337 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:39.330+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:39.341 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:39.334+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:39.357 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:39.350+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:39.529 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:39.523+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:39.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:39.808+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:39.817 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:39.808+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:39.944 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:39.937+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:40.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:40.351+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:40.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:40.372+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:40.413 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:40.405+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:40.537 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:40.531+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:40.853 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:40.846+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:40.866 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:40.860+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:40.906 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:40.899+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:41.382 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:41.374+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:41.398 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:41.392+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:41.408 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:41.402+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:41.532 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:41.526+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:41.840 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:41.832+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:41.856 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:41.849+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:41.901 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:41.894+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:42.371 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:42.365+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:42.394 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:42.387+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:42.394 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:42.387+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:42.568 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:42.561+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:46:42.879 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:46:42.871+0000 1694c700 -1 osd.7 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:30.676742+0000 front 2019-11-29T17:45:25.318115+0000 (oldest deadline 2019-11-29T17:46:10.567058+0000) 2019-11-29T17:46:42.901 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:46:42.893+0000 1694c700 -1 osd.6 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:25.237579+0000 front 2019-11-29T17:45:30.676344+0000 (oldest deadline 2019-11-29T17:46:10.442873+0000) 2019-11-29T17:46:42.902 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:46:42.894+0000 1694c700 -1 osd.4 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.256818+0000 front 2019-11-29T17:45:27.341468+0000 (oldest deadline 2019-11-29T17:46:13.136735+0000) 2019-11-29T17:46:43.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:46:43.346+0000 1694c700 -1 osd.2 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.102445+0000 front 2019-11-29T17:45:27.085927+0000 (oldest deadline 2019-11-29T17:46:12.969211+0000) 2019-11-29T17:46:43.422 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:46:43.415+0000 1694c700 -1 osd.3 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:29.622852+0000 front 2019-11-29T17:45:29.792854+0000 (oldest deadline 2019-11-29T17:46:11.303812+0000) 2019-11-29T17:46:43.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:46:43.443+0000 1694c700 -1 osd.5 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:26.856372+0000 front 2019-11-29T17:45:27.091611+0000 (oldest deadline 2019-11-29T17:46:12.737036+0000) 2019-11-29T17:46:43.480 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:46:43.474+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 47sec e353 v353) 2019-11-29T17:46:43.622 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:46:43.616+0000 1694c700 -1 osd.1 353 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:45:27.976746+0000 front 2019-11-29T17:45:27.969277+0000 (oldest deadline 2019-11-29T17:46:11.456425+0000) 2019-11-29T17:47:08.681 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:21.569 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.684 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:21.572 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.725 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:21.612 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.730 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:21.616 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.771 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:21.543 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.776 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:21.549 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.870 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:21.757 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.949 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:21.835 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:08.969 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:21.741 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:17.373 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:17.354+0000 33a70700 -1 osd.2 357 build_incremental_map_msg missing incremental map 357 2019-11-29T17:47:17.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:17.388+0000 38279700 -1 osd.2 357 build_incremental_map_msg missing incremental map 357 2019-11-29T17:47:17.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:17.525+0000 39a7c700 -1 osd.2 357 build_incremental_map_msg missing incremental map 357 2019-11-29T17:47:17.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:17.535+0000 38279700 -1 osd.2 357 build_incremental_map_msg missing incremental map 357 2019-11-29T17:47:17.654 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:17.586+0000 35a74700 -1 osd.2 357 build_incremental_map_msg missing incremental map 357 2019-11-29T17:47:19.794 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:32.682 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.796 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:32.683 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.838 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:32.610 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.842 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:32.615 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.843 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:34:32.730 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.850 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:32.737 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.854 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:32.741 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.891 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:34:32.663 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:19.910 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:34:32.797 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:47:41.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:41.517+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:42.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:42.503+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:43.113 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:47:43.098+0000 20e22700 -1 RGWWatcher::handle_error cookie 5488904704 err (107) Transport endpoint is not connected 2019-11-29T17:47:43.483 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:43.475+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:43.850 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:43.842+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:43.913 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:43.902+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:44.493 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:44.485+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:44.870 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:44.862+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:44.947 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:44.940+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:45.011 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:45.004+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:45.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:45.502+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:45.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:45.867+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:45.975 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:45.969+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:45.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:45.977+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:46.480 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:46.472+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:46.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:46.493+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:46.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:46.909+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:47.027 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:47.019+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:47.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:47.029+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:47.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:47.500+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:47.528 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:47.519+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:47.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:47.955+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:48.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:48.004+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:48.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:48.079+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:48.528 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:48.521+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:48.547 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:48.540+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:48.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:48.910+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:49.033 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:49.027+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:49.096 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:49.089+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:49.356 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:49.350+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:49.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:49.498+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:49.580 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:49.566+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:49.932 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:49.925+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:50.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:50.035+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:50.090 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:50.081+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:50.357 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:50.350+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:50.486 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:50.479+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:50.537 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:50.529+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:50.936 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:50.929+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:51.064 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:51.056+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:51.108 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:51.102+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:51.389 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:51.357+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:51.478 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:51.467+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:51.494 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:51.487+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:51.897 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:51.889+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:52.073 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:52.066+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:52.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:52.080+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:52.384 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:52.378+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:52.488 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:52.481+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:52.540 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:52.532+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:52.878 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:52.870+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:53.073 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:53.066+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:53.131 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:53.124+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:53.363 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:53.357+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:53.506 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:53.498+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:53.548 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:53.541+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:53.923 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:53.914+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:54.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:54.062+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:54.148 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:54.140+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:54.411 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:54.404+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:54.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:54.487+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:54.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:54.503+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:54.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:54.895+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:55.113 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:55.106+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:55.211 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:55.192+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:55.461 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:55.455+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:55.483 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:55.474+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:55.489 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:55.481+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:55.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:55.879+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:56.131 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:56.123+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:56.203 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:56.196+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:56.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:56.421+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:47:56.432 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:56.426+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:56.450 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:56.442+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:56.472 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:56.465+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:56.912 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:56.904+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:57.140 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:57.134+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:57.218 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:57.211+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:57.453 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:57.444+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:47:57.455 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:57.448+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:57.471 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:57.464+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:57.483 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:57.476+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:57.939 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:57.931+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:58.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:58.142+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:58.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:58.172+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:58.432 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:58.424+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:58.452 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:58.444+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:58.497 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:58.483+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:47:58.498 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:58.487+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:58.905 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:58.895+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:47:59.170 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:47:59.164+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:47:59.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:47:59.174+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:47:59.477 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:47:59.469+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:47:59.488 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:59.482+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:47:59.489 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:47:59.482+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:47:59.491 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:47:59.485+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:47:59.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:47:59.904+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:00.161 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:00.155+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:00.234 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:00.226+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:00.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:00.454+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:00.495 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:00.488+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:00.499 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:00.493+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:00.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:00.502+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:00.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:00.953+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:01.170 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:01.162+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:01.267 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:01.260+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:01.439 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:01.431+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:01.464 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:01.458+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:01.471 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:01.465+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:01.543 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:01.535+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:01.941 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:01.932+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:02.199 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:02.191+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:02.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:02.290+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:02.467 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:02.452+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:02.521 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:02.514+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:02.522 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:02.516+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:02.524 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:02.516+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:02.822 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:48:02.816+0000 20e22700 -1 RGWWatcher::handle_error cookie 5417367424 err (107) Transport endpoint is not connected 2019-11-29T17:48:02.969 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:02.961+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:03.213 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:03.205+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:03.321 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:03.314+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:03.436 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:03.426+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:03.497 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:03.488+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:03.556 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:03.541+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:03.562 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:03.545+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:03.986 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:03.978+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:04.250 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:04.242+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:04.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:04.293+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:04.406 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:04.398+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:04.538 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:04.530+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:04.585 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:04.577+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:04.587 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:04.579+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:04.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:04.950+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:05.223 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:05.215+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:05.325 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:05.318+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:05.419 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:05.410+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:05.571 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:05.563+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:05.601 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:05.591+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:05.604 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:05.598+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:05.948 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:05.940+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:06.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:06.203+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:06.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:06.319+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:06.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:06.372+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:06.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:06.567+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:06.575 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:06.569+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:06.604 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:06.596+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:06.945 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:06.937+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:07.238 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:07.222+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:07.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:07.293+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:07.425 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:07.411+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:07.623 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:07.612+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:07.626 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:07.617+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:07.634 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:07.626+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:07.907 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:07.898+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:08.202 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:08.197+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:08.277 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:08.268+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:08.277 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:08.270+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:08.426 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:08.417+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:08.595 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:08.586+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:08.667 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:08.660+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:08.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:08.662+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:08.932 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:08.918+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:09.198 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:09.191+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:09.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:09.300+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:09.308 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:09.302+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:09.383 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:09.374+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:09.630 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:09.621+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:09.650 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:09.644+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:09.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:09.649+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:09.907 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:09.894+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:10.181 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:10.174+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:10.264 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:10.256+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:10.272 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:10.265+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:10.358 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:10.344+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:10.657 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:10.650+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:10.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:10.679+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:10.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:10.683+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:10.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:10.868+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:11.172 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:11.166+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:11.281 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:11.274+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:11.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:11.277+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:11.399 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:11.391+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:11.673 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:11.663+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:11.673 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:11.666+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:11.704 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:11.697+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:11.869 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:11.862+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:12.144 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:12.137+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:12.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:12.307+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:12.315 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:12.310+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:12.450 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:12.438+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:12.644 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:12.638+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:12.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:12.640+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:12.677 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:12.670+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:12.843 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:12.836+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:13.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:13.127+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:13.302 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:13.296+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:13.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:13.300+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:13.481 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:13.470+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:13.658 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:13.652+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:13.663 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:13.655+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:13.663 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:13.657+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:13.749 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:13.742+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:13.896 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:13.890+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:14.105 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:14.098+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:14.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:14.276+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:14.284 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:14.279+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:14.496 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:14.484+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:14.647 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:14.641+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:14.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:14.646+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:14.708 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:14.696+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:14.742 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:14.734+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:14.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:14.918+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:15.135 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:15.125+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:15.303 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:15.297+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:15.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:15.300+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:15.514 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:15.506+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:15.677 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:15.670+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:15.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:15.690+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:15.698 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:15.693+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:15.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:15.758+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:15.980 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:15.973+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:16.159 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:16.153+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:16.348 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:16.341+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:16.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:16.346+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:16.535 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:16.527+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:16.670 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:16.662+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:16.713 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:16.703+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:16.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:16.708+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:16.757 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:16.749+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:17.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:17.001+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:17.130 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:17.124+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:17.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:17.378+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:17.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:17.381+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:17.518 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:17.510+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:17.646 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:17.638+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:17.736 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:17.729+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:17.737 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:17.731+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:17.743 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:17.735+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:18.027 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:18.020+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:18.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:18.107+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:18.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:18.340+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:18.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:18.345+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:18.545 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:18.538+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:18.621 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:18.613+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:18.713 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:18.706+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:18.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:18.708+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:18.758 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:18.751+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:18.817 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:48:18.809+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e358 v358) 2019-11-29T17:48:19.077 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:19.071+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:19.078 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:19.070+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:19.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:19.356+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:19.364 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:19.359+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:19.599 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:19.592+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:19.674 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:19.667+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:19.717 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:19.710+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:19.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:19.713+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:19.777 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:19.769+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:20.080 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:20.066+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:20.234 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:20.098+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:20.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:20.380+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:20.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:20.383+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:20.632 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:20.624+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:20.683 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:20.675+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:20.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:20.707+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:20.715 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:20.709+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:20.779 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:20.771+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:21.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:21.069+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:21.108 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:21.100+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:21.369 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:21.364+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:21.370 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:21.366+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:21.598 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:21.589+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:21.697 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:21.689+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:21.720 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:21.714+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:21.725 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:21.719+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:21.785 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:21.778+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:22.080 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:22.071+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:22.118 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:22.109+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:22.354 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:22.347+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:22.356 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:22.350+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:22.650 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:22.642+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:22.706 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:22.699+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:22.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:22.701+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:22.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:22.739+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:22.771 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:22.764+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:23.070 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:23.065+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:23.121 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:23.113+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:23.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:23.357+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:23.365 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:23.360+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:23.645 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:23.638+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:23.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:23.730+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:23.740 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:23.734+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:23.783 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:23.775+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:23.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:23.779+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:23.836 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:48:23.830+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e358 v358) 2019-11-29T17:48:24.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:24.037+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:24.163 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:24.155+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:24.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:24.349+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:24.356 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:24.351+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:24.668 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:24.659+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:24.696 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:24.689+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:24.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:24.692+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:24.774 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:24.767+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:24.776 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:24.769+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:25.034 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:25.028+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:25.153 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:25.146+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:25.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:25.343+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:25.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:25.345+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:25.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:25.629+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:25.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:25.700+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:25.709 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:25.703+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:25.780 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:25.771+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:25.808 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:25.800+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:26.079 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:26.072+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:26.133 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:26.126+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:26.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:26.320+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:26.329 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:26.323+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:26.670 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:26.662+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:26.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:26.665+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:26.677 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:26.669+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:26.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:26.809+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:26.826 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:26.818+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:27.124 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:27.117+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:27.161 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:27.149+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:27.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:27.289+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:27.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:27.290+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:27.633 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:27.625+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:27.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:27.628+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:27.655 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:27.647+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:27.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:27.794+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:27.832 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:27.824+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:28.104 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:28.098+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:28.206 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:48:28.198+0000 1694c700 -1 osd.5 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:57.950308+0000 front 2019-11-29T17:46:58.089501+0000 (oldest deadline 2019-11-29T17:47:43.225548+0000) 2019-11-29T17:48:28.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:28.260+0000 1694c700 -1 osd.3 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.981731+0000 front 2019-11-29T17:46:59.787103+0000 (oldest deadline 2019-11-29T17:47:43.851723+0000) 2019-11-29T17:48:28.268 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:28.263+0000 1694c700 -1 osd.3 358 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:28.594 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:28.589+0000 1694c700 -1 osd.0 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:47:15.758259+0000 (oldest deadline 2019-11-29T17:47:55.758259+0000) 2019-11-29T17:48:28.596 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:48:28.591+0000 1694c700 -1 osd.0 358 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e357) 2019-11-29T17:48:28.661 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:48:28.654+0000 1694c700 -1 osd.7 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.707225+0000 front 2019-11-29T17:46:59.951998+0000 (oldest deadline 2019-11-29T17:47:45.572426+0000) 2019-11-29T17:48:28.789 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:48:28.776+0000 1694c700 -1 osd.4 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:47:00.295273+0000 front 2019-11-29T17:47:01.040351+0000 (oldest deadline 2019-11-29T17:48:12.907213+0000) 2019-11-29T17:48:28.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:48:28.832+0000 1694c700 -1 osd.6 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:56.491175+0000 front 2019-11-29T17:46:56.682491+0000 (oldest deadline 2019-11-29T17:47:41.160607+0000) 2019-11-29T17:48:28.907 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:48:28.897+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e358 v358) 2019-11-29T17:48:29.079 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:48:29.074+0000 1694c700 -1 osd.2 358 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:46:59.429139+0000 front 2019-11-29T17:46:59.427962+0000 (oldest deadline 2019-11-29T17:47:44.076823+0000) 2019-11-29T17:48:29.441 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:29.356+0000 1694c700 -1 osd.3 359 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:30.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:48:30.346+0000 1694c700 -1 osd.3 360 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:106769 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5417367424] snapc 0=[] ondisk+write+known_if_redirected e358) 2019-11-29T17:48:53.508 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:48:53.499+0000 3927b700 -1 osd.1 363 build_incremental_map_msg missing incremental map 363 2019-11-29T17:48:53.771 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:48:53.750+0000 34271700 -1 osd.1 363 build_incremental_map_msg missing incremental map 363 2019-11-29T17:48:53.831 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:48:53.807+0000 34271700 -1 osd.1 363 build_incremental_map_msg missing incremental map 363 2019-11-29T17:48:53.843 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:48:53.821+0000 38a7a700 -1 osd.1 363 build_incremental_map_msg missing incremental map 363 2019-11-29T17:48:53.851 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:48:53.846+0000 37a78700 -1 osd.1 363 build_incremental_map_msg missing incremental map 363 2019-11-29T17:49:22.367 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:49:22.336+0000 1694c700 -1 osd.0 364 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=71 ondisk+retry+read+known_if_redirected e361) 2019-11-29T17:49:23.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:49:23.415+0000 1694c700 -1 osd.0 364 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=71 ondisk+retry+read+known_if_redirected e361) 2019-11-29T17:50:15.367 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:15.360+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:15.539 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:15.533+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:16.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:16.380+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:16.586 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:16.577+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:17.390 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:17.382+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:17.609 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:17.602+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:17.971 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:17.964+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:18.381 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:18.373+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:18.527 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:18.520+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:18.655 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:18.647+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:18.973 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:18.964+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:19.237 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:19.229+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:19.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:19.422+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:19.501 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:19.494+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:19.514 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:19.508+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:19.703 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:19.695+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:19.934 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:19.926+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:20.279 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:20.271+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:20.402 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:20.394+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:20.514 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:20.508+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:20.552 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:20.538+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:20.696 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:20.688+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:20.742 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:20.733+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:20.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:20.895+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:21.314 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:21.306+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:21.359 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:21.351+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:21.500 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:21.493+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:21.505 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:21.499+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:21.680 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:21.673+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:21.781 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:21.774+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:21.920 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:21.912+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:22.302 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:22.294+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:22.399 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:22.391+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:22.490 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:22.482+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:22.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:22.498+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:22.640 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:22.632+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:22.820 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:22.811+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:22.895 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:22.886+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:23.340 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:23.332+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:23.445 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:23.434+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:23.483 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:23.475+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:23.542 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:23.535+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:23.604 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:23.596+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:23.809 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:23.801+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:23.895 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:23.888+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:24.381 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:24.373+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:24.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:24.409+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:24.515 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:24.507+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:24.528 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:24.522+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:24.585 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:24.574+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:24.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:24.780+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:24.940 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:24.933+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:25.376 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:25.367+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:25.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:25.425+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:25.518 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:25.513+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:25.523 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:25.518+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:25.612 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:25.602+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:25.760 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:25.752+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:25.921 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:25.914+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:26.353 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:26.345+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:26.410 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:26.402+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:26.478 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:26.472+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:26.509 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:26.503+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:26.636 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:26.629+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:26.768 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:26.760+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:26.971 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:26.964+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:27.318 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:27.311+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:27.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:27.384+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:27.452 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:27.446+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:27.503 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:27.498+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:27.620 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:27.610+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:27.747 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:27.739+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:27.941 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:27.934+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:28.277 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:28.269+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:28.375 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:28.367+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:28.462 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:28.456+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:28.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:28.462+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:28.603 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:28.594+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:28.783 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:28.774+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:28.987 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:28.982+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:29.238 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:29.228+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:29.399 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:29.391+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:29.486 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:29.481+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:29.504 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:29.499+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:29.608 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:29.601+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:29.777 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:29.769+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:30.009 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:30.002+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:30.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:30.253+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:30.386 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:30.379+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:30.454 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:30.447+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:30.540 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:30.534+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:30.637 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:30.630+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:30.760 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:30.749+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:31.053 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:31.048+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:31.280 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:31.270+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:31.362 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:31.354+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:31.417 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:31.412+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:31.530 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:31.525+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:31.608 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:31.600+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:31.789 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:31.781+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:32.070 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:32.064+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:32.256 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:32.247+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:32.350 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:32.341+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:32.455 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:32.449+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:32.487 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:32.481+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:32.627 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:32.616+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:32.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:32.816+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:33.031 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:33.025+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:33.267 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:33.258+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:33.375 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:33.367+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:33.453 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:33.447+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:33.480 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:33.475+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:33.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:33.659+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:33.836 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:33.827+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:34.012 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:34.006+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:34.287 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:34.279+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:34.369 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:34.360+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:34.451 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:34.443+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:34.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:34.453+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:34.673 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:34.663+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:34.880 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:34.871+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:35.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:35.045+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:35.283 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:35.274+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:35.368 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:35.360+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:35.490 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:35.484+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:35.502 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:35.496+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:35.725 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:35.716+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:35.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:35.922+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:36.052 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:36.046+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:36.284 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:36.275+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:36.377 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:36.368+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:36.447 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:36.441+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:36.521 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:36.513+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:36.700 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:36.690+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:36.953 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:36.945+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:37.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:36.999+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:37.290 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:37.282+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:37.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:37.384+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:37.439 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:37.432+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:37.541 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:37.534+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:37.726 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:37.718+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:37.963 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:37.957+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:37.982 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:37.974+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:38.345 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:38.336+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:38.383 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:38.374+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:38.457 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:38.451+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:38.572 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:38.564+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:38.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:38.763+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:38.961 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:38.954+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:38.997 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:38.989+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:39.377 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:39.368+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:39.436 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:39.427+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:39.470 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:39.464+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:39.564 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:39.558+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:39.812 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:39.804+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:39.972 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:39.963+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:40.001 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:39.994+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:40.368 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:40.359+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:40.410 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:40.401+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:40.478 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:40.471+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:40.608 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:40.601+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:40.870 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:40.860+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:40.950 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:40.941+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:40.959 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:40.950+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:41.335 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:41.326+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:41.438 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:41.429+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:41.510 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:41.503+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:41.580 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:41.573+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:41.897 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:41.889+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:41.912 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:41.905+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:41.925 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:41.918+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:42.351 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:42.343+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:42.472 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:42.464+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:42.502 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:42.495+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:42.539 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:42.531+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:42.914 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:42.906+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:42.924 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:42.915+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:42.927 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:42.919+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:43.317 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:43.309+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:43.512 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:43.504+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:43.517 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:43.510+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:43.537 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:43.531+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:43.889 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:43.880+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:43.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:43.958+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:43.970 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:43.964+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:44.365 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:44.353+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:44.471 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:44.463+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:44.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:44.486+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:44.512 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:44.506+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:44.881 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:44.871+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:44.937 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:44.929+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:44.990 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:44.984+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:45.377 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:45.370+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:45.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:45.475+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:45.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:45.505+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:45.526 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:45.519+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:45.886 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:45.878+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:45.917 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:45.909+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:45.959 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:45.953+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:46.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:46.379+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:46.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:46.485+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:46.524 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:46.516+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:46.532 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:46.526+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:46.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:46.881+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:46.895 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:46.888+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:46.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:46.985+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:47.353 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:47.345+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:47.504 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:47.496+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:47.539 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:47.533+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:47.566 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:47.561+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:47.875 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:47.866+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:47.881 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:47.873+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:48.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:48.024+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:48.346 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:48.338+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:48.550 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:48.542+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:50:48.594 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:50:48.584+0000 1694c700 -1 osd.3 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.496487+0000 front 2019-11-29T17:49:34.498234+0000 (oldest deadline 2019-11-29T17:50:18.238987+0000) 2019-11-29T17:50:48.624 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:50:48.602+0000 1694c700 -1 osd.1 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.489310+0000 front 2019-11-29T17:49:34.126312+0000 (oldest deadline 2019-11-29T17:50:18.809834+0000) 2019-11-29T17:50:48.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:50:48.910+0000 1694c700 -1 osd.5 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:30.994482+0000 front 2019-11-29T17:49:34.780120+0000 (oldest deadline 2019-11-29T17:50:14.744131+0000) 2019-11-29T17:50:48.921 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:50:48.912+0000 1694c700 -1 osd.6 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.753013+0000 front 2019-11-29T17:49:34.686485+0000 (oldest deadline 2019-11-29T17:50:19.898230+0000) 2019-11-29T17:50:49.077 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:50:49.071+0000 1694c700 -1 osd.2 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.422489+0000 front 2019-11-29T17:49:34.725464+0000 (oldest deadline 2019-11-29T17:50:17.710103+0000) 2019-11-29T17:50:49.329 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:50:49.321+0000 1694c700 -1 osd.7 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:34.415057+0000 front 2019-11-29T17:49:34.501016+0000 (oldest deadline 2019-11-29T17:50:18.989855+0000) 2019-11-29T17:50:49.455 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:50:49.446+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e364 v364) 2019-11-29T17:50:49.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:50:49.517+0000 1694c700 -1 osd.4 364 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:49:33.084889+0000 front 2019-11-29T17:49:33.374823+0000 (oldest deadline 2019-11-29T17:50:14.753733+0000) 2019-11-29T17:51:17.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:17.069+0000 28259700 -1 reset not still connected to 0x101f64c0 2019-11-29T17:51:17.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:17.069+0000 28259700 -1 reset not still connected to 0x14a9a380 2019-11-29T17:51:19.818 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:38:32.706 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:19.836 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:38:32.721 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:19.837 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:38:32.724 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:19.837 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:38:32.610 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:19.839 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:38:32.726 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:19.843 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:38:32.615 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:20.116 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:38:32.889 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:20.257 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:38:33.144 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:20.264 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:38:33.152 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:51:47.811 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:47.803+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:48.459 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:48.450+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:48.583 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:48.576+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:48.764 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:48.754+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:48.787 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:48.777+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:48.863 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:48.852+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:49.465 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:49.455+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:49.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:49.623+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:49.783 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:49.764+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:49.797 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:49.790+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:49.811 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:49.803+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:49.825 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:49.816+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:50.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:50.451+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:50.599 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:50.591+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:50.758 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:50.749+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:50.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:50.801+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:50.835 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:50.826+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:50.858 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:50.850+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:51.161 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:51.154+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:51.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:51.484+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:51.613 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:51.605+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:51.767 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:51.757+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:51.812 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:51.800+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:51.813 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:51.801+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:51.873 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:51.865+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:52.121 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:52.115+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:52.542 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:52.534+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:52.594 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:52.588+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:52.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:52.759+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:52.796 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:52.786+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:52.811 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:52.802+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:52.900 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:52.893+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:53.080 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:53.073+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:53.582 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:53.573+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:53.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:53.580+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:53.772 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:53.764+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:53.811 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:53.803+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:53.863 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:53.854+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:53.928 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:53.920+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:54.041 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:54.035+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:54.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:54.585+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:54.632 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:54.622+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:54.802 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:54.793+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:54.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:54.837+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:54.850 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:54.841+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:54.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:54.906+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:55.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:55.036+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:55.597 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:55.590+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:55.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:55.616+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:55.853 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:55.844+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:55.868 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:55.857+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:55.910 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:55.900+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:55.943 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:55.935+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:56.092 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:56.086+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:56.607 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:56.600+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:56.674 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:56.665+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:56.847 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:56.838+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:56.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:56.862+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:56.872 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:56.863+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:56.901 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:56.895+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:57.116 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:57.111+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:57.638 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:57.631+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:57.640 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:57.632+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:57.841 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:57.833+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:57.862 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:57.854+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:57.903 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:57.895+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:57.922 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:57.917+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:58.087 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:58.080+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:51:58.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:58.083+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:58.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:58.595+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:58.653 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:58.645+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:58.844 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:58.835+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:58.881 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:58.874+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:58.887 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:58.880+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:58.943 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:58.935+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:51:59.058 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:59.052+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:51:59.059 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:51:59.054+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:51:59.586 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:51:59.579+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:51:59.664 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:51:59.655+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:51:59.840 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:51:59.831+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:51:59.921 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:51:59.914+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:51:59.933 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:51:59.925+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:51:59.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:51:59.932+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:00.053 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:00.046+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:00.054 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:00.048+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:00.599 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:00.593+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:00.694 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:00.687+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:00.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:00.814+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:00.879 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:00.872+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:00.912 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:00.903+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:00.974 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:00.966+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:01.070 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:01.063+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:01.070 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:01.065+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:01.648 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:01.642+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:01.733 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:01.725+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:01.806 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:01.798+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:01.840 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:01.832+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:01.939 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:01.930+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:01.970 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:01.962+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:02.032 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:02.025+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:02.033 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:02.027+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:02.672 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:02.665+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:02.751 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:02.742+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:02.828 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:02.820+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:02.870 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:02.863+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:02.899 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:02.891+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:02.950 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:02.943+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:02.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:02.990+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:03.000 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:02.994+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:03.671 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:03.664+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:03.739 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:03.730+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:03.796 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:03.789+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:03.861 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:03.853+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:03.921 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:03.915+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:03.957 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:03.949+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:03.982 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:03.974+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:03.984 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:03.977+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:04.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:04.661+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:04.721 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:04.703+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:04.766 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:52:04.733+0000 20e22700 -1 RGWWatcher::handle_error cookie 821004400 err (107) Transport endpoint is not connected 2019-11-29T17:52:04.826 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:04.818+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:04.834 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:04.827+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:04.949 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:04.941+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:04.957 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:04.951+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:04.996 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:04.990+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:04.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:04.992+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:05.670 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:05.661+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:05.693 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:05.685+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:05.812 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:05.805+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:05.891 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:05.881+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:05.967 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:05.959+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:05.967 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:05.961+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:06.033 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:06.026+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:06.037 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:06.031+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:06.676 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:06.667+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:06.681 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:06.674+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:06.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:06.782+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:06.849 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:06.841+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:06.925 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:06.916+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:06.937 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:06.929+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:06.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:06.990+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:06.997 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:06.992+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:07.708 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:07.700+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:07.708 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:07.701+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:07.752 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:07.744+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:07.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:07.834+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:07.963 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:07.957+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:07.968 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:07.960+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:08.034 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:08.025+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:08.034 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:08.026+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:08.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:08.691+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:08.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:08.753+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:08.766 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:08.758+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:08.843 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:08.835+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:08.937 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:08.929+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:08.939 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:08.933+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:09.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:09.076+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:09.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:09.078+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:09.707 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:09.696+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:09.733 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:09.725+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:09.811 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:09.798+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:09.811 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:09.800+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:09.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:09.895+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:09.909 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:09.904+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:09.913 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:09.907+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:10.113 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:10.108+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:10.115 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:10.110+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:10.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:10.721+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:10.755 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:10.747+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:10.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:10.799+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:10.829 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:10.821+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:10.902 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:10.892+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:10.937 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:10.928+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:10.937 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:10.931+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:11.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:11.113+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:11.124 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:11.118+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:11.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:11.751+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:11.762 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:11.756+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:11.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:11.783+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:11.798 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:11.791+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:11.876 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:11.869+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:11.916 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:11.909+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:11.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:11.914+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:12.144 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:12.133+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:12.144 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:12.137+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:12.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:12.717+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:12.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:12.783+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:12.792 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:12.782+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:12.807 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:12.801+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:12.883 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:12.875+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:12.951 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:12.944+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:12.952 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:12.947+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:13.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:13.151+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:13.159 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:13.153+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:13.701 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:13.693+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:13.773 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:13.764+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:13.831 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:13.824+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:13.841 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:13.833+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:13.880 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:13.865+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:13.992 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:13.986+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:13.994 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:13.990+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:14.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:14.188+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:14.198 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:14.192+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:14.670 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:14.662+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:14.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:14.778+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:14.789 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:14.784+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:14.844 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:14.836+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:14.862 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:14.854+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:14.977 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:14.971+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:14.978 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:14.973+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:15.159 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:15.149+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:15.162 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:15.152+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:15.670 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:15.662+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:15.760 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:15.751+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:15.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:15.801+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:15.860 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:15.828+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:15.877 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:15.869+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:16.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:16.013+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:16.023 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:16.018+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:16.135 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:16.129+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:16.138 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:16.132+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:16.647 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:16.633+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:16.779 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:16.773+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:16.799 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:16.791+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:16.878 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:16.869+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:16.928 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:16.918+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:17.017 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:17.011+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:17.018 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:17.012+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:17.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:17.152+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:17.160 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:17.155+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:17.641 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:17.633+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:17.771 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:17.763+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:17.797 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:17.792+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:17.879 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:17.871+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:17.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:17.955+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:18.036 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:18.028+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:18.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:18.031+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:18.159 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:18.153+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:18.160 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:18.154+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:18.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:18.625+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:18.793 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:18.778+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:18.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:18.841+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:18.923 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:18.909+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:19.009 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:19.001+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:19.048 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:19.042+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:19.049 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:19.043+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:19.151 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:19.144+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:19.152 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:19.146+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:19.618 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:19.606+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:19.744 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:52:19.735+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 41sec e370 v370) 2019-11-29T17:52:19.762 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:19.754+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:19.895 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:19.889+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:19.931 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:19.920+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:19.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:19.985+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:20.041 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:20.034+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:20.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:20.037+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:20.184 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:20.176+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:20.186 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:20.180+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:20.581 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:20.570+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:20.782 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:20.773+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:20.868 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:20.859+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:20.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:20.877+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:20.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:20.951+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:21.018 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:21.012+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:21.020 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:21.013+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:21.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:21.160+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:21.170 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:21.164+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:21.541 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:21.533+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:21.788 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:21.778+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:21.883 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:21.876+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:21.912 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:21.905+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:21.995 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:21.987+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:22.015 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:22.008+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:22.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:22.010+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:22.145 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:22.138+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:22.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:22.142+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:22.523 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:22.515+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:22.801 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:22.794+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:22.925 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:22.918+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:22.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:22.958+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:23.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:22.996+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:23.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:23.006+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:23.015 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:23.008+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:23.128 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:23.121+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:23.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:23.125+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:23.548 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:23.538+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:23.790 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:23.782+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:23.896 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:23.885+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:23.994 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:23.986+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:24.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:24.019+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:24.042 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:24.035+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:24.043 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:24.036+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:24.124 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:24.108+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:24.125 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:24.114+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:24.591 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:24.583+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:24.752 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:24.744+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:24.758 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:52:24.751+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 41sec e370 v370) 2019-11-29T17:52:24.918 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:24.910+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:24.991 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:24.984+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:25.032 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:25.025+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:25.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:25.040+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:25.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:25.041+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:25.097 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:25.090+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:25.097 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:25.091+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:25.627 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:25.619+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:25.710 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:25.702+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:25.873 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:25.865+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:26.007 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:25.998+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:26.053 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:26.045+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:26.085 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:26.076+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:26.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:26.079+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:26.094 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:26.084+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:26.095 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:26.088+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:26.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:26.611+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:26.727 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:26.718+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:26.846 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:26.837+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:26.989 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:26.980+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:27.066 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:27.057+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:27.067 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:27.061+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:27.070 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:27.063+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:27.071 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:27.065+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:27.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:27.083+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:27.611 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:27.602+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:27.739 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:27.729+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:27.889 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:27.877+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:27.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:27.947+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:28.064 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:28.057+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:28.093 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:28.086+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:28.094 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:28.087+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:28.115 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:28.107+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:28.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:28.111+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:28.664 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:28.655+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:28.751 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:28.742+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:28.932 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:28.923+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:28.937 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:28.929+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:29.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:29.092+0000 1694c700 -1 osd.0 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:51:18.027194+0000 (oldest deadline 2019-11-29T17:51:58.027194+0000) 2019-11-29T17:52:29.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:29.094+0000 1694c700 -1 osd.0 370 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:29.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:52:29.108+0000 1694c700 -1 osd.2 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:03.130098+0000 front 2019-11-29T17:51:02.337785+0000 (oldest deadline 2019-11-29T17:51:48.212688+0000) 2019-11-29T17:52:29.134 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:29.127+0000 1694c700 -1 osd.3 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.620451+0000 front 2019-11-29T17:51:04.065826+0000 (oldest deadline 2019-11-29T17:51:49.352563+0000) 2019-11-29T17:52:29.137 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:29.129+0000 1694c700 -1 osd.3 370 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:29.630 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:52:29.622+0000 1694c700 -1 osd.7 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.480054+0000 front 2019-11-29T17:51:04.348552+0000 (oldest deadline 2019-11-29T17:51:47.594803+0000) 2019-11-29T17:52:29.716 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:52:29.707+0000 1694c700 -1 osd.6 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.955192+0000 front 2019-11-29T17:51:04.854671+0000 (oldest deadline 2019-11-29T17:51:48.305330+0000) 2019-11-29T17:52:29.791 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:52:29.783+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 41sec e370 v370) 2019-11-29T17:52:29.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:52:29.896+0000 1694c700 -1 osd.4 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:04.757758+0000 front 2019-11-29T17:51:04.835488+0000 (oldest deadline 2019-11-29T17:51:47.032099+0000) 2019-11-29T17:52:29.976 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:52:29.955+0000 1694c700 -1 osd.5 370 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:51:06.816390+0000 front 2019-11-29T17:51:06.996056+0000 (oldest deadline 2019-11-29T17:51:48.368764+0000) 2019-11-29T17:52:30.439 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:52:30.376+0000 1694c700 -1 osd.0 371 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:107135 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e368) 2019-11-29T17:52:30.472 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:30.394+0000 1694c700 -1 osd.3 371 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:31.528 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:52:31.511+0000 1694c700 -1 osd.3 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:107164 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 821004400] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-11-29T17:52:58.993 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:52:58.978+0000 33a70700 -1 osd.1 375 build_incremental_map_msg missing incremental map 375 2019-11-29T17:52:59.390 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:52:59.291+0000 35a74700 -1 osd.1 375 build_incremental_map_msg missing incremental map 375 2019-11-29T17:52:59.437 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:52:59.330+0000 37a78700 -1 osd.1 375 build_incremental_map_msg missing incremental map 375 2019-11-29T17:52:59.450 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:52:59.367+0000 3a27d700 -1 osd.1 375 build_incremental_map_msg missing incremental map 375 2019-11-29T17:52:59.510 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:52:59.478+0000 36275700 -1 osd.1 375 build_incremental_map_msg missing incremental map 375 2019-11-29T17:53:00.113 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:40:13.000 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.147 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:40:13.033 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.213 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:40:13.100 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.220 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:40:13.105 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.242 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:40:13.015 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.249 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:40:13.022 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.295 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:40:13.180 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.351 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:40:13.238 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:53:00.413 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:40:13.185 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:54:18.970 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:18.964+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:19.247 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:19.240+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:19.548 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:19.539+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:20.001 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:19.993+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:20.214 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:20.208+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:20.549 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:20.540+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:20.937 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:20.929+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:21.052 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:21.046+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:21.232 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:21.226+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:21.503 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:21.494+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:21.506 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:21.498+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:21.664 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:21.655+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:21.913 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:21.906+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:22.054 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:22.048+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:22.237 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:22.227+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:22.464 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:22.454+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:22.504 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:22.495+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:22.666 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:22.657+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:22.920 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:22.913+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:23.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:23.059+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:23.242 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:23.235+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:23.455 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:23.444+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:23.548 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:23.539+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:23.685 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:23.676+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:23.778 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:23.768+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:23.914 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:23.908+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:24.106 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:24.099+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:24.207 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:24.201+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:24.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:24.493+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:24.549 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:24.539+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:24.689 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:24.680+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:24.761 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:24.752+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:24.892 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:24.885+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:25.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:25.145+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:25.225 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:25.218+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:25.515 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:25.505+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:25.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:25.509+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:25.693 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:25.684+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:25.744 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:25.735+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:25.932 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:25.925+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:26.105 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:26.098+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:26.193 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:26.186+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:26.510 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:26.502+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:26.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:26.558+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:26.706 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:26.697+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:26.716 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:26.708+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:26.932 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:26.924+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:27.081 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:27.073+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:27.211 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:27.204+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:27.553 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:27.541+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:27.553 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:27.543+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:27.715 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:27.705+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:27.719 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:27.709+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:27.905 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:27.898+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:28.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:28.032+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:28.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:28.181+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:28.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:28.558+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:28.606 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:28.594+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:28.699 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:28.689+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:28.776 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:28.760+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:28.952 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:28.944+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:29.053 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:29.046+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:29.166 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:29.158+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:29.550 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:29.542+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:29.562 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:29.555+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:29.655 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:29.646+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:29.801 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:29.793+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:30.002 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:29.990+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:30.078 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:30.071+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:30.128 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:30.121+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:30.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:30.498+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:30.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:30.587+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:30.663 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:30.654+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:30.763 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:30.754+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:31.023 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:31.016+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:31.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:31.092+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:31.158 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:31.152+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:31.521 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:31.507+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:31.557 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:31.549+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:31.683 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:31.675+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:31.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:31.794+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:32.051 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:32.044+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:32.132 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:32.125+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:32.140 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:32.134+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:32.539 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:32.531+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:32.542 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:32.534+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:32.685 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:32.677+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:32.807 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:32.799+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:33.090 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:33.084+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:33.104 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:33.099+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:33.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:33.133+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:33.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:33.543+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:33.581 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:33.574+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:33.734 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:33.726+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:33.836 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:33.820+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:34.056 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:34.049+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:34.116 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:34.109+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:34.156 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:34.149+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:34.581 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:34.572+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:34.610 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:34.602+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:34.797 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:34.787+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:34.823 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:34.815+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:35.014 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:35.007+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:35.099 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:35.094+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:35.148 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:35.141+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:35.546 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:35.538+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:35.590 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:35.580+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:35.820 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:35.812+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:35.835 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:35.827+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:35.984 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:35.978+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:36.085 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:36.078+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:36.103 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:36.095+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:36.590 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:36.583+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:36.598 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:36.590+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:36.816 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:36.808+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:36.887 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:36.880+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:36.985 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:36.979+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:37.074 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:37.068+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:37.113 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:37.106+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:37.551 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:37.543+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:37.559 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:37.552+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:37.811 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:37.803+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:37.869 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:37.861+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:37.962 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:37.955+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:38.050 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:38.044+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:38.137 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:38.131+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:38.530 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:38.522+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:38.614 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:38.607+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:38.832 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:38.824+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:38.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:38.848+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:39.007 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:38.999+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:39.044 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:39.036+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:39.158 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:39.151+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:39.559 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:39.551+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:39.641 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:39.633+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:39.864 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:39.856+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:39.896 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:39.888+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:40.027 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:40.019+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:40.064 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:40.055+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:40.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:40.134+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:40.561 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:40.554+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:40.691 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:40.682+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:40.873 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:40.864+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:40.905 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:40.897+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:41.031 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:41.019+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:41.068 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:41.059+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:41.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:41.160+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:41.549 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:41.541+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:41.692 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:41.684+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:41.894 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:41.885+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:41.904 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:41.895+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:42.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:42.011+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:42.087 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:42.078+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:42.175 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:42.165+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:42.513 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:42.505+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:42.660 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:42.651+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:42.876 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:42.868+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:42.885 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:42.875+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:43.046 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:43.035+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:43.106 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:43.098+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:43.132 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:43.121+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:43.483 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:43.475+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:43.688 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:43.680+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:43.841 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:43.833+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:43.877 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:43.869+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:44.079 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:44.064+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:44.138 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:44.130+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:44.155 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:44.147+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:44.534 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:44.526+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:44.736 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:44.728+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:44.800 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:44.792+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:44.833 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:44.824+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:45.123 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:45.115+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:45.155 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:45.147+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:45.208 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:45.200+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:45.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:45.507+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:45.788 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:45.779+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:45.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:45.794+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:45.830 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:45.821+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:46.084 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:46.076+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:46.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:46.163+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:46.211 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:46.204+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:46.474 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:46.466+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:46.845 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:46.827+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:46.849 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:46.838+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:46.874 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:46.866+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:47.111 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:47.103+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:47.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:47.162+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:47.253 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:47.245+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:47.449 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:47.441+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:47.799 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:47.790+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:47.867 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:47.859+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:47.917 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:47.909+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:48.085 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:48.077+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:48.140 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:48.133+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:48.230 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:48.221+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:48.429 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:48.421+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:48.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:48.814+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:48.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:48.895+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:48.920 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:48.912+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:49.129 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:49.116+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:49.192 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:49.185+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:49.247 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:49.238+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:49.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:49.474+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:49.804 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:49.796+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:49.888 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:49.880+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:49.908 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:49.901+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:50.099 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:50.089+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:50.195 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:50.187+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:50.204 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:50.193+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:50.402 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:54:50.396+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e375 v375) 2019-11-29T17:54:50.444 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:50.435+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:50.856 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:50.843+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:50.892 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:50.885+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:50.963 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:50.954+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:51.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:51.043+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:51.196 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:51.187+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:51.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:51.231+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:51.471 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:51.461+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:51.850 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:51.831+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:51.932 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:51.924+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:51.991 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:51.983+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:52.101 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:52.094+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:52.167 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:52.149+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:52.206 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:52.186+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:52.454 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:52.442+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:52.804 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:52.796+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:52.967 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:52.959+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:53.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:53.030+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:53.109 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:53.101+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:53.122 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:53.114+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:53.189 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:53.176+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:53.419 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:53.409+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:53.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:53.793+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:53.972 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:53.962+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:53.996 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:53.988+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:54.138 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:54.127+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:54.147 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:54.137+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:54.217 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:54.210+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:54.455 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:54.447+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:54.775 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:54:54.768+0000 1694c700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:40.042589+0000 front 2019-11-29T17:53:40.248582+0000 (oldest deadline 2019-11-29T17:54:23.274054+0000) 2019-11-29T17:54:54.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:54:54.927+0000 1694c700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:39.554708+0000 front 2019-11-29T17:53:38.442427+0000 (oldest deadline 2019-11-29T17:54:21.116504+0000) 2019-11-29T17:54:54.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:54:54.951+0000 1694c700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.464205+0000 front 2019-11-29T17:53:34.476789+0000 (oldest deadline 2019-11-29T17:54:19.502408+0000) 2019-11-29T17:54:55.123 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:54:55.116+0000 1694c700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.473635+0000 front 2019-11-29T17:53:34.776239+0000 (oldest deadline 2019-11-29T17:54:19.135504+0000) 2019-11-29T17:54:55.183 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:54:55.175+0000 1694c700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:34.364986+0000 front 2019-11-29T17:53:34.363425+0000 (oldest deadline 2019-11-29T17:54:20.235557+0000) 2019-11-29T17:54:55.240 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:54:55.233+0000 1694c700 -1 osd.3 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:33.071776+0000 front 2019-11-29T17:53:40.246937+0000 (oldest deadline 2019-11-29T17:54:18.776094+0000) 2019-11-29T17:54:55.411 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:54:55.403+0000 1694c700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:53:35.203328+0000 front 2019-11-29T17:53:35.205350+0000 (oldest deadline 2019-11-29T17:54:20.937664+0000) 2019-11-29T17:54:55.484 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:54:55.464+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e375 v375) 2019-11-29T17:55:21.728 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:34.615 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.735 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:34.622 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.746 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:34.633 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.749 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:34.636 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.758 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:34.530 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.764 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:34.536 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.902 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:34.789 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.930 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:34.817 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:21.990 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:34.762 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.674 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:43.562 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.697 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:43.585 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.710 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:43.597 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.713 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:43.600 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.727 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:42:43.615 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.747 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:42:43.634 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.755 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:43.528 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.759 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:43.532 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:30.820 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:42:43.593 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:55:54.615 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:54.607+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:54.617 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:54.610+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:54.806 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:54.799+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:54.914 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:54.909+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:55:55.620 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:55.611+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:55.624 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:55.616+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:55.849 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:55.840+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:55.911 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:55.905+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:55:56.607 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:56.599+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:56.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:56.630+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:56.811 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:56.803+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:56.887 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:56.881+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:55:56.888 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:56.882+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:55:57.336 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:55:57.325+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:55:57.507 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:55:57.500+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:55:57.573 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:57.564+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:57.626 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:57.618+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:57.848 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:57.841+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:57.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:57.923+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:55:57.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:57.928+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:55:58.214 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:55:58.209+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:55:58.306 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:55:58.298+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:55:58.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:55:58.542+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:55:58.587 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:58.579+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:58.622 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:58.613+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:58.819 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:58.811+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:58.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:58.926+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:55:58.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:58.928+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:55:59.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:55:59.185+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:55:59.298 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:55:59.290+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:55:59.554 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:55:59.547+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:55:59.613 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:55:59.603+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:55:59.617 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:55:59.608+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:55:59.793 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:55:59.784+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:55:59.955 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:59.948+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:55:59.955 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:55:59.950+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:00.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:00.141+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:00.263 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:00.255+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:00.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:00.559+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:00.582 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:00.571+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:00.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:00.589+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:00.763 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:00.755+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:00.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:00.977+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:00.986 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:00.980+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:01.111 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:01.105+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:01.289 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:01.281+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:01.528 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:01.520+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:01.607 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:01.600+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:01.619 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:01.613+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:01.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:01.796+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:01.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:01.941+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:01.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:01.943+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:02.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:02.140+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:02.331 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:02.320+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:02.534 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:02.521+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:02.608 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:02.600+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:02.620 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:02.612+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:02.799 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:02.790+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:02.992 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:02.986+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:02.993 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:02.988+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:03.135 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:03.129+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:03.356 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:03.349+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:03.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:03.477+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:03.616 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:03.608+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:03.657 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:03.649+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:03.830 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:03.822+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:03.990 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:03.984+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:03.992 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:03.988+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:04.154 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:04.148+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:04.394 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:04.386+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:04.469 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:04.460+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:04.594 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:04.586+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:04.644 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:04.637+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:04.809 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:04.801+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:04.972 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:04.962+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:04.972 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:04.964+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:05.198 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:05.190+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:05.199 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:05.193+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:05.423 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:05.415+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:05.447 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:05.439+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:05.646 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:05.627+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:05.665 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:05.658+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:05.780 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:05.771+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:05.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:05.941+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:05.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:05.943+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:06.180 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:06.173+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:06.181 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:06.175+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:06.401 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:06.392+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:06.403 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:06.395+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:06.625 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:06.610+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:06.703 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:06.696+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:06.790 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:06.782+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:06.952 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:06.946+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:06.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:06.948+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:07.190 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:07.184+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:07.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:07.187+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:07.359 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:07.351+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:07.443 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:07.436+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:07.651 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:07.643+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:07.693 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:07.685+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:07.801 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:07.792+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:07.957 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:07.950+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:07.958 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:07.952+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:08.152 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:08.146+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:08.153 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:08.148+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:08.367 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:08.359+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:08.457 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:08.445+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:08.657 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:08.650+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:08.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:08.681+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:08.809 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:08.801+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:08.988 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:08.980+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:08.988 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:08.982+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:09.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:09.153+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:09.160 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:09.155+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:09.376 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:09.366+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:09.427 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:09.419+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:09.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:09.680+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:09.694 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:09.687+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:09.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:09.796+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:10.033 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:10.025+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:10.049 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:10.041+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:10.125 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:10.117+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:10.126 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:10.119+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:10.427 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:10.418+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:10.477 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:10.469+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:10.667 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:10.659+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:10.715 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:10.708+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:10.840 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:10.825+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:11.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:11.058+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:11.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:11.060+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:11.122 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T17:56:11.117+0000 20e22700 -1 RGWWatcher::handle_error cookie 5292816448 err (107) Transport endpoint is not connected 2019-11-29T17:56:11.128 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:11.122+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:11.130 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:11.125+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:11.443 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:11.435+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:11.491 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:11.483+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:11.637 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:11.624+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:11.748 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:11.739+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:11.847 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:11.839+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:12.054 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:12.046+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:12.055 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:12.048+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:12.122 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:12.113+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:12.123 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:12.116+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:12.444 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:12.436+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:12.529 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:12.520+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:12.630 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:12.620+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:12.741 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:12.734+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:12.816 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:12.808+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:13.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:13.089+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:13.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:13.091+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:13.155 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:13.148+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:13.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:13.152+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:13.456 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:13.447+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:13.511 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:13.502+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:13.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:13.645+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:13.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:13.721+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:13.835 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:13.826+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:14.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:14.049+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:14.059 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:14.052+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:14.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:14.161+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:14.172 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:14.165+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:14.471 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:14.463+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:14.503 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:14.494+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:14.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:14.677+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:14.692 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:14.683+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:14.875 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:14.866+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:15.074 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:15.066+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:15.076 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:15.068+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:15.177 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:15.167+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:15.178 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:15.171+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:15.494 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:15.485+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:15.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:15.547+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:15.693 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:15.685+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:15.709 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:15.702+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:15.867 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:15.859+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:16.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:16.110+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:16.120 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:16.114+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:16.207 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:16.199+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:16.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:16.201+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:16.522 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:16.513+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:16.539 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:16.525+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:16.718 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:16.709+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:16.732 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:16.723+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:16.847 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:16.837+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:17.098 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:17.088+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:17.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:17.090+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:17.213 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:17.204+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:17.215 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:17.207+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:17.535 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:17.526+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:17.543 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:17.534+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:17.735 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:17.723+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:17.767 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:17.759+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:17.868 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:17.860+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:18.146 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:18.137+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:18.147 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:18.139+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:18.235 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:18.226+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:18.239 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:18.232+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:18.507 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:18.497+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:18.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:18.509+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:18.724 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:18.715+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:18.757 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:18.750+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:18.924 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:18.914+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:19.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:19.142+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:19.150 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:19.143+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:19.243 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:19.232+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:19.245 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:19.235+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:19.478 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:19.469+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:19.520 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:56:19.512+0000 1694c700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.981956+0000 front 2019-11-29T17:55:12.134257+0000 (oldest deadline 2019-11-29T17:55:57.256079+0000) 2019-11-29T17:56:19.691 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:19.682+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:19.804 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:19.793+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:19.932 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:19.923+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:20.177 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:20.170+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:20.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:20.172+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:20.263 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:20.254+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:20.264 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:20.256+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:20.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:20.498+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:20.651 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:20.643+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:20.765 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:20.757+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:20.986 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:20.978+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:21.148 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:21.141+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:21.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:21.142+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:21.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:21.252+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:21.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:21.255+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:21.498 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:21.489+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:21.715 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:21.698+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:21.772 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:21.765+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:21.950 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:21.941+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:22.130 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:22.109+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:22.130 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:22.112+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:22.230 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:22.221+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:22.234 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:22.224+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:22.541 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:22.534+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:22.731 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:22.724+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:22.783 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:22.775+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:22.925 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:22.909+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:23.151 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:23.142+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:23.151 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:23.145+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:23.226 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:23.217+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:23.228 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:23.220+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:23.574 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:23.566+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:23.698 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:23.689+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:23.829 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:23.821+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:23.927 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:56:23.919+0000 1694c700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:09.632624+0000 front 2019-11-29T17:55:09.541909+0000 (oldest deadline 2019-11-29T17:55:54.228398+0000) 2019-11-29T17:56:24.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:24.176+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:24.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:24.179+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:24.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:24.250+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:24.260 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:24.253+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:24.625 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:24.617+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:24.691 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:24.684+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:24.873 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:24.865+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:25.228 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:25.220+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:25.229 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:25.222+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:25.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:25.230+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:25.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:25.232+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:25.656 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:25.648+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:25.720 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:25.712+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:25.842 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:25.833+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:26.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:26.183+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:26.196 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:26.188+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:26.209 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:26.202+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:26.211 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:26.205+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:26.686 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:26.678+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:26.709 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:26.701+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:26.869 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:26.861+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:27.173 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:27.164+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:27.174 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:27.167+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:27.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:27.231+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:27.241 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:27.234+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:27.685 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:27.677+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:27.716 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:27.707+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:27.919 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:27.911+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:28.225 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:28.216+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:28.226 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:28.219+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:28.278 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:28.271+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:28.282 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:28.275+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:28.644 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:28.635+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:28.695 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:28.688+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:28.920 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:28.912+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:29.199 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:29.192+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:29.202 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:29.197+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:29.332 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:29.326+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:29.336 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:29.330+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:29.681 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:29.673+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:29.696 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:29.688+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:29.958 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:29.951+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:30.217 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:30.211+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:30.218 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:30.212+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:30.305 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:30.298+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:30.306 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:30.300+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:30.709 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:30.701+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:30.716 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:30.703+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:30.825 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:56:30.814+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e381 v381) 2019-11-29T17:56:30.958 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:30.952+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:31.237 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:31.230+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:31.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:31.232+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:31.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:31.291+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:31.301 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:31.294+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:31.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:31.663+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:31.769 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:31.744+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:31.922 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:31.916+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:32.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:32.259+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:32.265 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:32.258+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:32.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:32.260+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:32.273 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:32.262+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:32.708 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:32.700+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:32.754 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:32.739+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:32.903 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:32.897+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:33.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:33.221+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:33.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:33.223+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:33.280 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:33.273+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:33.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:33.275+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:33.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:33.665+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:33.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:33.770+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:33.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:33.896+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:34.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:34.237+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:34.244 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:34.239+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:34.276 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:34.270+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:34.277 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:34.271+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:34.658 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:34.649+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:34.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:34.741+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:34.922 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:34.916+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:35.282 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:35.275+0000 1694c700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:55:24.244139+0000 (oldest deadline 2019-11-29T17:56:04.244139+0000) 2019-11-29T17:56:35.287 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:35.281+0000 1694c700 -1 osd.0 381 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e379) 2019-11-29T17:56:35.309 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:35.303+0000 1694c700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:11.975231+0000 front 2019-11-29T17:55:12.001881+0000 (oldest deadline 2019-11-29T17:55:56.611816+0000) 2019-11-29T17:56:35.312 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:35.307+0000 1694c700 -1 osd.3 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:35.673 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:56:35.665+0000 1694c700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.601248+0000 front 2019-11-29T17:55:10.529301+0000 (oldest deadline 2019-11-29T17:55:54.536788+0000) 2019-11-29T17:56:35.717 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:56:35.709+0000 1694c700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:10.995160+0000 front 2019-11-29T17:55:11.089730+0000 (oldest deadline 2019-11-29T17:55:54.245041+0000) 2019-11-29T17:56:35.892 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:56:35.885+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e381 v381) 2019-11-29T17:56:35.908 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:56:35.902+0000 1694c700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:55:12.701133+0000 front 2019-11-29T17:55:12.711323+0000 (oldest deadline 2019-11-29T17:55:57.343835+0000) 2019-11-29T17:56:36.360 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T17:56:36.337+0000 1694c700 -1 osd.0 382 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:107542 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e379) 2019-11-29T17:56:36.625 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:36.556+0000 1694c700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:56:37.676 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:56:37.655+0000 1694c700 -1 osd.3 383 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80974 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e378) 2019-11-29T17:58:26.188 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:26.181+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:26.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:26.292+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:27.158 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:27.149+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:27.281 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:27.273+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:27.296 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:27.287+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:27.352 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:27.344+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:27.381 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:27.372+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:28.121 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:28.113+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:28.176 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:28.167+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:28.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:28.280+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:28.316 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:28.307+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:28.353 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:28.340+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:28.392 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:28.384+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:29.127 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:29.120+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:29.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:29.139+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:29.216 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:29.207+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:29.285 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:29.277+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:29.331 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:29.323+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:29.343 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:29.335+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:29.411 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:29.404+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:30.143 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:30.134+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:30.146 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:30.139+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:30.227 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:30.219+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:30.298 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:30.289+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:30.337 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:30.329+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:30.389 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:30.376+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:30.389 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:30.374+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:31.120 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:31.111+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:31.165 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:31.157+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:31.237 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:31.228+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:31.306 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:31.299+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:31.340 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:31.332+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:31.348 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:31.341+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:31.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:31.344+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:32.126 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:32.118+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:32.212 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:32.203+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:32.218 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:32.210+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:32.347 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:32.338+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:32.358 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:32.350+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:32.367 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:32.359+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:32.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:32.379+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:33.098 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:33.090+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:33.175 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:33.167+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:33.241 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:33.232+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:33.318 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:33.310+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:33.374 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:33.364+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:33.380 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:33.373+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:33.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:33.391+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:34.103 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:34.095+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:34.144 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:34.137+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:34.276 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:34.262+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:34.313 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:34.304+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:34.362 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:34.353+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:34.425 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:34.417+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:34.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:34.421+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:35.123 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:35.115+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:35.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:35.129+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:35.317 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:35.308+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:35.320 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:35.312+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:35.339 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:35.330+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:35.398 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:35.390+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:35.403 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:35.394+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:36.161 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:36.152+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:36.189 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:36.182+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:36.297 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:36.287+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:36.304 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:36.295+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:36.379 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:36.370+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:36.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:36.376+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:36.398 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:36.391+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:37.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:37.128+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:37.215 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:37.207+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:37.305 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:37.292+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:37.311 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:37.301+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:37.389 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:37.381+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:37.395 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:37.388+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:37.423 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:37.415+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:38.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:38.156+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:38.196 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:38.187+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:38.261 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:38.253+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:38.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:38.263+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:38.420 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:38.412+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:38.422 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:38.414+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:38.440 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:38.431+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:39.218 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:39.209+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:39.229 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:39.221+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:39.235 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:39.227+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:39.286 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:39.277+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:39.393 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:39.376+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:39.432 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:39.423+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:39.471 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:39.462+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:40.208 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:40.200+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:40.258 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:40.250+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:40.268 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:40.259+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:40.287 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:40.278+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:40.358 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:40.350+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:40.430 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:40.420+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:40.493 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:40.484+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:41.222 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:41.213+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:41.227 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:41.219+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:41.275 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:41.268+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:41.326 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:41.319+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:41.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:41.339+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:41.403 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:41.396+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:41.453 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:41.446+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:42.239 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:42.231+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:42.273 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:42.266+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:42.312 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:42.304+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:42.334 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:42.327+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:42.345 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:42.338+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:42.388 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:42.376+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:42.434 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:42.427+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:43.205 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:43.197+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:43.287 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:43.280+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:43.301 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:43.289+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:43.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:43.295+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:43.350 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:43.341+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:43.410 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:43.403+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:43.437 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:43.428+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:44.222 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:44.213+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:44.283 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:44.275+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:44.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:44.298+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:44.317 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:44.307+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:44.318 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:44.310+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:44.394 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:44.386+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:44.473 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:44.464+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:45.255 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:45.245+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:45.279 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:45.272+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:45.284 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:45.275+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:45.289 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:45.280+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:45.289 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:45.283+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:45.377 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:45.370+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:45.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:45.473+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:46.267 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:46.258+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:46.278 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:46.268+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:46.285 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:46.279+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:46.319 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:46.309+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:46.321 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:46.316+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:46.342 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:46.336+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:46.468 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:46.460+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:47.254 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:47.248+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:47.260 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:47.249+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:47.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:47.274+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:47.293 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:47.285+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:47.336 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:47.329+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:47.362 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:47.352+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:47.487 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:47.477+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:48.208 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:48.202+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:48.264 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:48.254+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:48.271 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:48.261+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:48.317 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:48.310+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:48.324 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:48.318+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:48.348 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:48.339+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:48.502 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:48.492+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:49.181 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:49.166+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:49.261 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:49.245+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:49.280 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:49.271+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:49.300 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:49.292+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:49.328 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:49.320+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:49.357 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:49.347+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:49.498 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:49.488+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:50.227 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:50.219+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:50.268 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:50.258+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:50.276 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:50.267+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:50.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:50.275+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:50.305 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:50.298+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:50.317 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:50.308+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:50.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:50.526+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:51.250 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:51.244+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:51.270 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:51.264+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:51.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:51.282+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:51.311 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:51.305+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:51.315 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:51.306+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:51.357 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:51.347+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:51.527 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:51.518+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:52.265 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:52.257+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:52.277 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:52.268+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:52.278 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:52.270+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:52.287 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:52.280+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:52.339 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:52.331+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:52.355 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:52.346+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:52.567 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:52.557+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:53.291 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:53.282+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:53.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:53.297+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:53.306 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:53.299+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:53.314 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:53.307+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:53.315 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:53.308+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:53.389 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:53.377+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:53.607 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:53.598+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:54.294 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:54.288+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:54.299 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:54.292+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:54.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:54.315+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:54.327 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:54.320+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:54.339 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:54.330+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:54.402 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:54.394+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:54.625 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:54.616+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:55.280 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:55.274+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:55.301 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:55.293+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:55.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:55.305+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:55.323 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:55.315+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:55.361 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:55.354+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:55.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:55.409+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:55.655 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:55.646+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:56.253 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:56.247+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:56.270 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:56.260+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:56.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:56.299+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:56.316 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:56.307+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:56.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:56.347+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:56.377 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:56.370+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:56.627 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:56.619+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:57.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:57.259+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:57.276 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:57.269+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:57.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:57.298+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:57.321 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:57.313+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:57.344 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:57.337+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:57.372 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:57.363+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:57.634 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:57.625+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:58.273 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:58.265+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:58.282 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:58.275+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:58.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:58.297+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:58.315 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:58.304+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:58.315 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:58.304+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:58.341 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:58.332+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:58.595 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:58.586+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:58:59.229 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:58:59.221+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:58:59.285 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:58:59.277+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:58:59.295 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:58:59.287+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:58:59.308 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:58:59.301+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:58:59.332 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:58:59.323+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:58:59.376 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:58:59.350+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:58:59.581 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:58:59.573+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:59:00.244 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:59:00.235+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:59:00.305 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:59:00.299+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:59:00.307 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:59:00.301+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:59:00.329 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:59:00.323+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:59:00.336 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:59:00.327+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:59:00.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:59:00.360+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:59:00.557 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T17:59:00.546+0000 1694c700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:40.839656+0000 front 2019-11-29T17:57:40.836603+0000 (oldest deadline 2019-11-29T17:58:26.429277+0000) 2019-11-29T17:59:01.265 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T17:59:01.259+0000 1694c700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.853539+0000 front 2019-11-29T17:57:42.880591+0000 (oldest deadline 2019-11-29T17:58:27.546225+0000) 2019-11-29T17:59:01.276 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T17:59:01.268+0000 1694c700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:44.838008+0000 front 2019-11-29T17:57:44.611560+0000 (oldest deadline 2019-11-29T17:58:26.066711+0000) 2019-11-29T17:59:01.316 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T17:59:01.306+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e387 v387) 2019-11-29T17:59:01.321 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:59:01.313+0000 1694c700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.791546+0000 front 2019-11-29T17:57:43.776681+0000 (oldest deadline 2019-11-29T17:58:28.374713+0000) 2019-11-29T17:59:01.341 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T17:59:01.333+0000 1694c700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.275335+0000 front 2019-11-29T17:57:43.357891+0000 (oldest deadline 2019-11-29T17:58:27.131308+0000) 2019-11-29T17:59:01.360 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T17:59:01.352+0000 1694c700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:42.700738+0000 front 2019-11-29T17:57:42.692285+0000 (oldest deadline 2019-11-29T17:58:26.023833+0000) 2019-11-29T17:59:01.364 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T17:59:01.357+0000 1694c700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T17:57:43.836502+0000 front 2019-11-29T17:57:43.838062+0000 (oldest deadline 2019-11-29T17:58:27.214761+0000) 2019-11-29T17:59:13.759 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:46:26.647 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:13.768 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:46:26.655 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:13.776 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:46:26.663 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:13.781 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:46:26.668 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:13.794 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:46:26.566 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:13.800 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:46:26.572 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:14.026 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:46:26.913 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:14.086 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:46:26.973 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:14.133 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:46:26.906 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T17:59:59.699 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T17:59:59.690+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:00.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:00.713+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:01.761 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:01.752+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:02.036 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:02.025+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:02.355 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:02.345+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:02.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:02.667+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:02.742 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:02.734+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:02.908 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:02.900+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:03.075 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:03.065+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:03.383 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:03.374+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:03.470 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:03.461+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:03.491 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:03.483+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:03.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:03.625+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:03.756 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:03.747+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:03.930 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:03.923+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:04.103 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:04.093+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:04.345 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:04.336+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:04.514 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:04.506+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:04.516 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:04.505+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:04.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:04.648+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:04.784 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:04.775+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:04.890 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:04.883+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:05.102 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:05.092+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:05.334 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:05.324+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:05.494 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:05.487+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:05.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:05.492+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:05.647 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:05.640+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:05.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:05.791+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:05.922 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:05.915+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:06.098 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:06.088+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:06.326 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:06.318+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:06.459 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:06.452+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:06.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:06.500+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:06.626 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:06.619+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:06.770 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:06.761+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:06.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:06.911+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:07.132 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:07.123+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:07.370 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:07.360+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:07.479 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:07.470+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:07.490 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:07.484+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:07.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:07.650+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:07.784 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:07.771+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:07.969 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:07.962+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:08.173 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:08.163+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:08.409 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:08.400+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:08.449 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:08.440+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:08.499 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:08.490+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:08.708 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:08.701+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:08.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:08.815+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:08.953 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:08.946+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:09.223 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:09.213+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:09.443 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:09.435+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:09.463 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:09.457+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:09.495 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:09.487+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:09.731 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:09.721+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:09.731 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:09.725+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:09.820 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:09.812+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:09.979 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:09.971+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:10.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:10.181+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:10.456 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:10.448+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:10.494 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:10.487+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:10.546 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:10.537+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:10.712 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:10.706+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:10.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:10.708+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:10.874 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:10.865+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:11.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:11.006+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:11.179 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:11.172+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:11.450 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:11.442+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:11.523 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:11.514+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:11.533 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:11.526+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:11.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:11.746+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:11.754 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:11.748+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:11.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:11.889+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:11.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:11.976+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:12.221 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:12.212+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:12.452 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:12.445+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:12.521 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:12.512+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:12.542 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:12.535+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:12.741 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:12.731+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:12.741 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:12.734+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:12.863 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:12.855+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:13.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:13.003+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:13.271 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:13.263+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:13.455 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:13.446+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:13.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:13.499+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:13.535 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:13.528+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:13.763 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:13.752+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:13.763 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:13.754+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:13.894 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:13.886+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:14.053 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:14.046+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:14.268 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:14.260+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:14.463 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:14.456+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:14.512 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:14.504+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:14.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:14.506+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:14.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:14.731+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:14.742 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:14.736+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:14.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:14.939+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:15.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:15.080+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:15.280 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:15.271+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:15.444 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:15.436+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:15.498 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:15.490+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:15.565 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:15.557+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:15.713 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:15.706+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:15.714 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:15.707+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:15.913 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:15.904+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:16.054 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:16.041+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:16.258 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:16.250+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:16.463 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:16.454+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:16.473 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:16.466+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:16.552 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:16.544+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:16.699 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:16.693+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:16.700 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:16.695+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:16.928 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:16.919+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:17.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:17.090+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:17.217 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:17.208+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:17.481 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:17.471+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:17.488 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:17.480+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:17.576 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:17.568+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:17.740 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:17.733+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:17.741 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:17.735+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:17.910 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:17.903+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:18.141 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:18.133+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:18.240 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:18.232+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:18.465 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:18.457+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:18.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:18.474+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:18.590 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:18.582+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:18.792 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:18.784+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:18.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:18.790+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:18.883 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:18.875+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:19.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:19.145+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:19.202 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:19.194+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:19.482 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:19.473+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:19.495 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:19.488+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:19.602 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:19.594+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:19.821 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:19.814+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:19.821 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:19.816+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:19.920 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:19.912+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:20.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:20.160+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:20.230 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:20.222+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:20.465 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:20.456+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:20.528 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:20.522+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:20.617 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:20.610+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:20.789 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:20.784+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:20.790 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:20.785+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:20.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:20.965+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:21.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:21.206+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:21.228 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:21.220+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:21.492 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:21.486+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:21.517 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:21.509+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:21.665 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:21.655+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:21.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:21.808+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:21.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:21.809+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:21.929 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:21.922+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:22.235 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:22.229+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:22.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:22.273+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:22.474 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:22.467+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:22.562 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:22.554+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:22.626 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:22.619+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:22.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:22.789+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:22.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:22.790+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:22.960 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:22.951+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:23.204 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:23.198+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:23.294 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:23.285+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:23.498 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:23.492+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:23.591 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:23.583+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:23.650 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:23.641+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:23.827 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:23.820+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:23.828 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:23.822+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:23.916 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:23.909+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:24.221 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:24.215+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:24.250 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:24.242+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:24.527 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:24.520+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:24.614 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:24.605+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:24.678 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:24.670+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:24.833 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:24.827+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:24.835 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:24.830+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:24.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:24.923+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:25.202 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:25.196+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:25.219 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:25.211+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:25.509 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:25.503+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:25.661 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:25.653+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:25.714 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:25.706+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:25.845 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:25.837+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:25.846 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:25.840+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:25.897 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:25.888+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:26.231 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:26.224+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:26.236 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:26.228+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:26.476 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:26.469+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:26.646 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:26.637+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:26.740 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:26.731+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:26.898 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:26.891+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:26.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:26.890+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:26.899 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:26.893+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:27.214 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:27.206+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:27.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:27.227+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:27.477 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:27.470+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:27.651 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:27.643+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:27.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:27.740+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:27.880 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:27.874+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:27.883 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:27.878+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:27.913 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:27.905+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:28.236 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:28.228+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:28.272 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:28.265+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:28.481 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:28.475+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:28.669 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:28.661+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:28.769 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:28.761+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:28.853 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:28.846+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:28.855 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:28.848+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:28.889 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:28.881+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:29.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:29.255+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:29.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:29.291+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:29.517 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:29.510+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:29.721 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:29.713+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:29.756 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:29.747+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:29.887 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:29.881+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:29.888 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:29.882+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:29.936 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:29.928+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:30.302 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:30.294+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:30.335 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:30.328+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:30.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:30.543+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:30.718 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:30.710+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:30.746 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:30.738+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:30.879 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:30.871+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:30.879 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:30.874+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:30.970 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:30.961+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:31.300 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:31.292+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:31.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:31.368+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:31.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:31.539+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:31.685 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:31.676+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:31.752 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:31.744+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:31.835 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:31.830+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:31.837 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:31.833+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:31.928 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:31.920+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:32.293 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:32.285+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:32.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:32.393+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:32.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:32.538+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:32.687 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:32.679+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:32.774 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:32.766+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:32.877 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:32.870+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:32.878 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:32.873+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:32.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:32.951+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:33.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:33.300+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:33.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:33.380+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:33.596 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:33.587+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:33.698 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:33.689+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:33.766 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:33.757+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:33.883 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:33.877+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:33.885 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:33.880+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:33.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:33.986+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:34.284 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:34.276+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:34.441 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:34.434+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:34.644 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:34.637+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:34.720 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:34.711+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:34.804 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:34.795+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:34.910 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:34.904+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:34.912 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:34.905+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:35.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:35.033+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:35.322 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:35.313+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:35.432 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:35.418+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:35.599 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:35.593+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:35.730 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:35.722+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:35.781 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:35.773+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:35.884 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:35.878+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:35.885 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:35.879+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:36.102 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:36.092+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:36.306 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:36.297+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:36.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:36.440+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:36.572 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:36.566+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:36.644 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:00:36.636+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e393 v393) 2019-11-29T18:00:36.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:36.730+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:36.763 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:36.755+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:36.845 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:36.838+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:36.847 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:36.842+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:37.072 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:37.064+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:37.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:37.253+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:37.456 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:37.449+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:37.562 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:37.555+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:37.738 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:37.729+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:37.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:37.794+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:37.894 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:37.887+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:37.899 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:37.893+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:38.081 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:38.072+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:38.271 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:38.262+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:38.500 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:38.495+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:38.529 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:38.522+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:38.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:38.741+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:38.815 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:38.806+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:38.921 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:38.915+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:38.922 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:38.918+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:39.077 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:39.066+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:39.317 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:39.308+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:39.495 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:39.489+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:39.575 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:39.569+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:39.781 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:39.771+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:39.800 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:39.792+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:39.956 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:39.949+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:39.957 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:39.951+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:40.053 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:40.044+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:40.326 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:40.316+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:40.467 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:40.457+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:40.565 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:40.557+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:40.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:40.740+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:40.829 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:40.818+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:40.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:40.907+0000 1694c700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T17:59:29.553572+0000 (oldest deadline 2019-11-29T18:00:09.553572+0000) 2019-11-29T18:00:40.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:40.908+0000 1694c700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:00:41.038 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:41.028+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:41.329 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:00:41.321+0000 1694c700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:15.458924+0000 front 2019-11-29T17:59:15.516813+0000 (oldest deadline 2019-11-29T18:00:01.354532+0000) 2019-11-29T18:00:41.500 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:00:41.492+0000 1694c700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.274900+0000 front 2019-11-29T17:59:18.371112+0000 (oldest deadline 2019-11-29T18:00:02.327713+0000) 2019-11-29T18:00:41.547 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:00:41.539+0000 1694c700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:18.478841+0000 front 2019-11-29T17:59:18.584298+0000 (oldest deadline 2019-11-29T18:00:03.170794+0000) 2019-11-29T18:00:41.750 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:00:41.742+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e393 v393) 2019-11-29T18:00:41.753 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:00:41.743+0000 1694c700 -1 osd.6 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.949727+0000 front 2019-11-29T17:59:17.933702+0000 (oldest deadline 2019-11-29T18:00:03.047449+0000) 2019-11-29T18:00:41.873 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:00:41.863+0000 1694c700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:17.962000+0000 front 2019-11-29T17:59:17.966959+0000 (oldest deadline 2019-11-29T18:00:02.035644+0000) 2019-11-29T18:00:41.992 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:00:41.983+0000 1694c700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T17:59:14.166639+0000 front 2019-11-29T17:59:14.164953+0000 (oldest deadline 2019-11-29T17:59:59.437234+0000) 2019-11-29T18:00:42.125 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:00:42.072+0000 1694c700 -1 osd.0 394 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e391) 2019-11-29T18:02:13.913 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:13.906+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:14.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:14.485+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:14.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:14.886+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:14.900 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:14.892+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:15.457 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:15.449+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:15.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:15.658+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:15.858 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:15.850+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:15.904 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:15.896+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:16.474 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:16.465+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:16.608 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:16.601+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:16.712 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:16.704+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:16.830 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:16.823+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:16.913 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:16.904+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:17.116 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:17.108+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:17.244 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:17.236+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:17.522 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:17.513+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:17.643 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:17.634+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:17.674 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:17.665+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:17.819 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:17.809+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:17.888 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:17.880+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:18.137 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:18.127+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:18.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:18.225+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:18.571 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:18.563+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:18.657 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:18.648+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:18.692 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:18.684+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:18.842 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:18.835+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:18.913 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:18.906+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:19.152 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:19.144+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:19.231 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:19.223+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:19.558 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:19.550+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:19.618 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:19.610+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:19.649 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:19.641+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:19.826 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:19.818+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:19.886 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:19.879+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:20.190 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:20.182+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:20.214 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:20.206+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:20.562 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:20.553+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:20.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:20.630+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:20.659 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:20.651+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:20.795 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:20.787+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:20.909 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:20.900+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:21.217 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:21.209+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:21.249 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:21.241+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:21.594 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:21.586+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:21.614 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:21.606+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:21.657 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:21.649+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:21.810 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:21.802+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:21.926 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:21.918+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:22.194 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:22.185+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:22.306 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:22.291+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:22.553 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:22.545+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:22.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:22.615+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:22.681 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:22.673+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:22.808 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:22.801+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:22.981 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:22.974+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:23.176 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:23.166+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:23.289 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:23.279+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:23.569 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:23.561+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:23.604 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:23.596+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:23.695 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:23.688+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:23.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:23.784+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:23.959 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:23.952+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:24.181 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:24.163+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:24.266 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:24.258+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:24.626 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:24.617+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:24.636 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:24.627+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:24.692 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:24.685+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:24.753 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:24.747+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:24.996 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:24.981+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:25.179 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:25.170+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:25.231 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:25.224+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:25.639 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:25.631+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:25.648 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:25.641+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:25.683 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:25.668+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:25.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:25.752+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:25.969 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:25.961+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:26.182 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:26.173+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:26.226 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:26.206+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:26.614 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:26.605+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:26.620 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:26.612+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:26.712 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:26.704+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:26.790 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:26.782+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:26.947 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:26.939+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:27.167 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:27.159+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:27.271 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:27.263+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:27.611 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:27.602+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:27.643 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:27.633+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:27.695 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:27.687+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:27.819 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:27.811+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:27.993 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:27.986+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:28.169 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:28.160+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:28.255 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:28.247+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:28.607 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:28.600+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:28.648 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:28.641+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:28.685 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:28.676+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:28.823 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:28.815+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:28.975 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:28.968+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:29.140 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:29.130+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:29.243 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:29.234+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:29.634 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:29.625+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:29.645 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:29.636+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:29.699 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:29.691+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:29.777 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:29.770+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:29.992 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:29.984+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:30.172 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:30.162+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:30.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:30.190+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:30.616 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:30.608+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:30.683 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:30.675+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:30.756 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:30.747+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:30.784 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:30.769+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:31.037 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:31.024+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:31.162 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:31.154+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:31.188 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:31.180+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:31.605 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:31.598+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:31.656 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:31.646+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:31.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:31.731+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:31.772 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:31.763+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:32.065 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:32.057+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:32.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:32.137+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:32.175 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:32.166+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:32.589 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:32.582+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:32.671 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:32.662+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:32.748 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:32.743+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:32.808 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:32.799+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:33.076 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:33.070+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:33.123 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:33.115+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:33.191 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:33.182+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:33.627 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:33.621+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:33.628 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:33.618+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:33.743 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:33.737+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:33.795 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:33.786+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:34.077 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:34.071+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:34.118 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:34.104+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:34.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:34.150+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:34.608 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:34.597+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:34.647 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:34.641+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:34.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:34.768+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:34.823 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:34.814+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:35.069 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:35.059+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:35.107 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:35.100+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:35.182 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:35.171+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:35.625 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:35.609+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:35.652 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:35.646+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:35.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:35.740+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:35.863 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:35.853+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:36.084 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:36.074+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:36.148 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:36.141+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:36.165 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:36.156+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:36.589 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:36.581+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:36.633 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:36.626+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:36.718 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:36.711+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:36.852 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:36.842+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:37.095 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:37.083+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:37.122 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:37.116+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:37.146 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:37.136+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:37.617 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:37.608+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:37.632 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:37.627+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:37.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:37.741+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:37.806 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:37.797+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:38.102 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:38.093+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:38.107 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:38.098+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:38.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:38.134+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:38.647 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:38.640+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:38.668 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:38.659+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:38.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:38.739+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:38.812 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:38.804+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:39.096 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:39.086+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:39.097 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:39.090+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:39.121 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:39.112+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:39.636 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:39.631+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:39.667 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:39.655+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:39.791 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:39.785+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:39.862 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:39.853+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:40.079 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:40.069+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:40.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:40.126+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:40.152 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:40.142+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:40.638 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:40.630+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:40.647 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:40.641+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:40.781 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:40.774+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:40.881 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:40.872+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:41.085 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:41.076+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:41.162 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:41.156+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:41.188 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:41.180+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:41.602 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:41.594+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:41.693 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:41.687+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:41.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:41.751+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:41.885 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:41.877+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:42.096 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:42.087+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:42.215 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:42.208+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:42.241 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:42.233+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:42.623 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:42.610+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:42.724 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:42.717+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:42.757 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:42.750+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:42.934 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:42.924+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:43.144 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:43.135+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:43.210 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:43.203+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:43.274 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:43.266+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:43.626 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:43.618+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:43.748 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:43.741+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:43.759 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:43.752+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:43.890 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:43.882+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:44.138 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:44.130+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:44.219 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:44.212+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:44.312 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:44.304+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:44.597 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:44.584+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:44.787 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:44.781+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:44.798 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:44.792+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:44.886 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:44.878+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:45.147 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:45.140+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:45.223 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:45.209+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:45.350 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:45.341+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:45.583 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:45.567+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:45.768 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:45.761+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:45.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:45.776+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:45.846 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:45.838+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:46.122 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:46.115+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:46.169 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:46.163+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:02:46.312 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:02:46.304+0000 1694c700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.053716+0000 front 2019-11-29T18:01:31.357905+0000 (oldest deadline 2019-11-29T18:02:16.901409+0000) 2019-11-29T18:02:46.588 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:02:46.577+0000 1694c700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.296166+0000 front 2019-11-29T18:01:33.479933+0000 (oldest deadline 2019-11-29T18:02:14.760271+0000) 2019-11-29T18:02:46.725 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:02:46.718+0000 1694c700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.630160+0000 front 2019-11-29T18:01:31.667823+0000 (oldest deadline 2019-11-29T18:02:16.478697+0000) 2019-11-29T18:02:46.773 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:02:46.766+0000 1694c700 -1 osd.3 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:31.382758+0000 front 2019-11-29T18:01:31.380404+0000 (oldest deadline 2019-11-29T18:02:14.654252+0000) 2019-11-29T18:02:46.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:02:46.863+0000 1694c700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.750096+0000 front 2019-11-29T18:01:30.816583+0000 (oldest deadline 2019-11-29T18:02:13.700320+0000) 2019-11-29T18:02:47.100 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:02:47.093+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e398 v398) 2019-11-29T18:02:47.107 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:02:47.099+0000 1694c700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.353804+0000 front 2019-11-29T18:01:33.045472+0000 (oldest deadline 2019-11-29T18:02:16.411040+0000) 2019-11-29T18:02:47.206 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:02:47.200+0000 1694c700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:01:33.488167+0000 front 2019-11-29T18:01:30.629081+0000 (oldest deadline 2019-11-29T18:02:13.371792+0000) 2019-11-29T18:03:16.506 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:50:29.394 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.528 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:50:29.403 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.528 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:50:29.414 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.531 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:50:29.419 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.534 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:50:29.306 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.550 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:50:29.321 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.790 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:50:29.668 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.790 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:50:29.675 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:16.860 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:50:29.630 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:44.364 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:03:44.206+0000 20e22700 -1 RGWWatcher::handle_error cookie 5435994336 err (107) Transport endpoint is not connected 2019-11-29T18:03:44.516 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:44.506+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:44.604 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:44.597+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:45.365 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:45.356+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:45.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:45.448+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:45.497 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:45.488+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:45.603 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:45.596+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:45.755 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:45.747+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:46.412 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:46.403+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:46.458 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:46.451+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:46.533 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:46.524+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:46.625 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:46.619+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:46.752 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:46.745+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:47.411 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:47.402+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:47.511 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:47.501+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:47.559 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:47.546+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:47.598 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:47.590+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:47.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:47.782+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:48.415 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:48.406+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:48.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:48.510+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:48.549 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:48.541+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:48.590 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:48.582+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:48.796 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:48.787+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:48.865 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:48.855+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:49.387 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:49.378+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:49.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:49.518+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:49.526 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:49.517+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:49.595 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:49.587+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:49.832 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:49.821+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:49.905 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:49.897+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:50.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:50.408+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:50.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:50.539+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:50.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:50.552+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:50.576 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:50.570+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:50.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:50.762+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:50.820 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:50.811+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:50.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:50.952+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:51.445 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:51.436+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:51.550 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:51.543+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:51.577 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:51.569+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:51.585 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:51.578+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:51.752 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:51.746+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:51.843 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:51.834+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:51.973 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:51.963+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:52.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:52.420+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:52.450 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:51:05.338 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.458 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:51:05.346 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.490 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:51:05.377 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.492 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:51:05.379 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.513 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:51:05.401 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.534 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:51:05.306 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.538 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:51:05.310 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.541 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:52.534+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:52.542 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:52.535+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:52.554 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:51:05.438 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.632 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:51:05.404 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:03:52.633 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:52.624+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:52.752 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:52.746+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:52.854 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:52.846+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:52.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:52.949+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:53.397 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:53.390+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:53.505 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:53.499+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:53.534 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:53.528+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:53.585 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:53.578+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:53.797 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:53.790+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:53.902 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:53.895+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:53.948 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:53.940+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:54.442 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:03:54.433+0000 1694c700 -1 osd.4 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.428294+0000 front 2019-11-29T18:03:03.281712+0000 (oldest deadline 2019-11-29T18:03:44.772542+0000) 2019-11-29T18:03:54.488 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:54.482+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:54.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:54.491+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:54.541 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:54.532+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:54.791 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:54.785+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:54.901 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:54.892+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:54.985 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:54.976+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:55.513 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:55.505+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:55.524 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:55.516+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:55.588 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:55.579+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:55.815 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:55.803+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:55.925 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:55.915+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:55.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:55.965+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:56.504 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:56.498+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:56.566 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:56.558+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:56.635 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:56.625+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:56.767 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:56.760+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:56.919 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:56.910+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:56.940 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:56.932+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:57.473 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:57.465+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:57.578 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:57.572+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:57.616 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:57.608+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:57.756 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:57.748+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:03:57.757 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:57.750+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:57.934 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:57.927+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:57.954 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:57.941+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:58.502 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:58.495+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:58.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:58.530+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:58.658 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:58.650+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:58.722 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:58.714+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:03:58.722 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:58.716+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:58.926 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:58.918+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:58.939 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:58.930+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:03:59.509 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:03:59.500+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:03:59.528 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:03:59.521+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:03:59.632 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:03:59.624+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:03:59.726 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:59.718+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:03:59.726 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:03:59.720+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:03:59.937 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:03:59.929+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:03:59.999 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:03:59.990+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:00.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:00.510+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:00.574 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:00.567+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:00.594 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:00.586+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:00.730 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:00.723+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:00.730 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:00.725+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:00.983 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:00.975+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:01.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:01.020+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:01.536 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:01.529+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:01.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:01.552+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:01.581 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:01.573+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:01.760 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:01.751+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:01.763 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:01.756+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:01.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:01.954+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:02.005 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:01.998+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:02.509 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:02.503+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:02.545 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:02.537+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:02.582 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:02.574+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:02.743 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:02.736+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:02.746 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:02.740+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:02.950 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:02.930+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:02.998 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:02.989+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:03.491 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:03.484+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:03.571 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:03.565+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:03.576 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:03.568+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:03.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:03.777+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:03.790 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:03.784+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:03.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:03.954+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:04.048 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:04.040+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:04.520 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:04.514+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:04.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:04.595+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:04.614 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:04.606+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:04.630 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:04:04.617+0000 20e22700 -1 RGWWatcher::handle_error cookie 387249280 err (107) Transport endpoint is not connected 2019-11-29T18:04:04.779 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:04.772+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:04.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:04.778+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:04.978 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:04.970+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:05.030 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:05.022+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:05.564 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:05.557+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:05.565 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:05.558+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:05.599 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:05.591+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:05.795 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:05.789+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:05.797 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:05.793+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:05.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:05.940+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:06.082 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:06.074+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:06.571 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:06.564+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:06.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:06.590+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:06.617 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:06.610+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:06.841 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:06.833+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:06.842 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:06.836+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:06.980 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:06.971+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:07.112 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:07.104+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:07.535 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:07.529+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:07.576 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:07.569+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:07.594 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:07.588+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:07.874 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:07.865+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:07.875 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:07.870+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:08.027 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:08.020+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:08.140 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:08.133+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:08.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:08.529+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:08.616 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:08.610+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:08.626 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:08.617+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:08.843 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:08.838+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:08.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:08.839+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:09.046 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:09.038+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:09.170 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:09.162+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:09.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:09.518+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:09.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:09.520+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:09.624 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:09.616+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:09.673 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:09.666+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:09.877 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:09.871+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:09.879 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:09.874+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:10.062 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:10.054+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:10.141 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:10.130+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:10.556 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:10.549+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:10.560 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:10.555+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:10.597 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:10.587+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:10.661 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:10.653+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:10.923 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:10.916+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:10.924 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:10.919+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:11.083 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:11.075+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:11.099 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:11.091+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:11.556 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:11.550+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:11.563 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:11.554+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:11.644 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:11.638+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:11.648 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:11.640+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:11.900 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:11.893+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:11.901 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:11.895+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:12.114 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:12.106+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:12.145 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:12.134+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:12.562 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:12.557+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:12.563 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:12.559+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:12.646 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:12.631+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:12.675 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:12.667+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:12.941 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:12.933+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:12.943 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:12.937+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:13.111 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:13.099+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:13.189 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:13.181+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:13.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:13.532+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:13.538 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:13.534+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:13.618 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:13.612+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:13.671 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:13.663+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:13.941 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:13.936+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:13.943 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:13.938+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:14.150 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:14.142+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:14.169 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:14.161+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:14.540 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:14.531+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:14.540 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:14.533+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:14.620 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:14.613+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:14.666 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:14.658+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:14.926 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:14.920+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:14.931 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:14.925+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:15.160 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:15.151+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:15.222 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:15.214+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:15.508 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:15.502+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:15.512 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:15.507+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:15.638 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:15.626+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:15.658 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:15.652+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:15.895 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:15.890+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:15.901 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:15.893+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:16.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:16.185+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:16.251 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:16.242+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:16.494 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:16.482+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:16.494 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:16.484+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:16.653 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:16.645+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:16.667 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:16.657+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:16.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:16.848+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:16.857 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:16.851+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:17.212 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:17.202+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:17.286 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:17.278+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:17.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:17.485+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:17.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:17.487+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:17.496 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:04:17.488+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e403 v403) 2019-11-29T18:04:17.639 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:17.628+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:17.662 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:17.655+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:17.901 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:17.892+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:17.901 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:17.893+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:18.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:18.176+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:18.320 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:18.312+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:18.452 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:18.444+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:18.452 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:18.447+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:18.661 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:18.654+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:18.686 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:18.673+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:18.892 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:18.886+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:18.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:18.891+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:19.228 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:19.217+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:19.285 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:19.277+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:19.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:19.468+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:19.477 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:19.472+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:19.678 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:19.670+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:19.678 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:19.666+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:19.864 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:19.856+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:19.865 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:19.859+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:20.254 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:20.245+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:20.295 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:20.286+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:20.448 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:20.441+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:20.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:20.444+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:20.647 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:20.639+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:20.715 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:20.708+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:20.897 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:20.890+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:20.899 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:20.893+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:21.262 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:21.255+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:21.293 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:21.285+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:21.436 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:21.429+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:21.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:21.431+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:21.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:21.630+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:21.746 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:21.741+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:21.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:21.889+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:21.896 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:21.891+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:22.263 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:22.254+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:22.275 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:22.264+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:22.477 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:22.471+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:22.479 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:22.474+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:22.511 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:04:22.505+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e403 v403) 2019-11-29T18:04:22.693 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:22.672+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:22.706 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:22.700+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:22.923 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:22.912+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:22.923 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:22.915+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:23.218 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:23.211+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:23.288 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:23.281+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:23.530 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:23.523+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:23.531 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:23.525+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:23.656 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:23.648+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:23.687 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:23.682+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:23.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:23.900+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:23.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:23.906+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:24.200 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:24.192+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:24.264 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:24.256+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:24.567 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:24.559+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:24.569 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:24.563+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:24.641 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:24.633+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:24.653 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:24.645+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:24.943 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:24.929+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:24.944 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:24.931+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:25.189 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:25.181+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:25.249 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:25.240+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:25.595 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:25.589+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:25.595 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:25.590+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:25.608 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:25.602+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:25.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:25.629+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:25.956 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:25.950+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:25.959 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:25.955+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:26.182 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:26.172+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:26.247 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:26.237+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:26.580 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:26.574+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:26.583 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:26.578+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:26.659 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:04:26.652+0000 1694c700 -1 osd.2 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.204262+0000 front 2019-11-29T18:03:00.200227+0000 (oldest deadline 2019-11-29T18:03:44.284746+0000) 2019-11-29T18:04:26.660 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:26.652+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:26.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:26.991+0000 1694c700 -1 osd.0 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:03:17.409717+0000 (oldest deadline 2019-11-29T18:03:57.409717+0000) 2019-11-29T18:04:27.001 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:26.995+0000 1694c700 -1 osd.0 403 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e402) 2019-11-29T18:04:27.211 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:04:27.197+0000 1694c700 -1 osd.6 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.272091+0000 front 2019-11-29T18:03:03.263092+0000 (oldest deadline 2019-11-29T18:03:44.893979+0000) 2019-11-29T18:04:27.212 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:04:27.200+0000 1694c700 -1 osd.5 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:03.246196+0000 front 2019-11-29T18:03:03.281642+0000 (oldest deadline 2019-11-29T18:03:47.912001+0000) 2019-11-29T18:04:27.555 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:04:27.536+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e403 v403) 2019-11-29T18:04:27.575 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:27.569+0000 1694c700 -1 osd.3 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:03:00.471141+0000 front 2019-11-29T18:03:00.518991+0000 (oldest deadline 2019-11-29T18:03:45.142146+0000) 2019-11-29T18:04:27.575 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:27.570+0000 1694c700 -1 osd.3 403 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:04:27.633 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:04:27.621+0000 1694c700 -1 osd.7 403 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:02:59.185323+0000 front 2019-11-29T18:02:59.123696+0000 (oldest deadline 2019-11-29T18:03:44.251418+0000) 2019-11-29T18:04:28.162 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:04:28.109+0000 1694c700 -1 osd.0 404 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80502 7.9 7.d101a989 (undecoded) ondisk+retry+write+known_if_redirected e402) 2019-11-29T18:04:28.547 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:04:28.542+0000 1694c700 -1 osd.3 404 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108339 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387249280 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-11-29T18:05:23.994 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:05:23.893+0000 1694c700 -1 osd.0 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108439 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e406) 2019-11-29T18:06:17.666 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:17.657+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:18.698 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:18.691+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:19.684 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:19.671+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:20.053 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:20.043+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:20.596 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:20.590+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:20.706 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:20.697+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:20.912 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:20.898+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:21.099 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:21.089+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:21.562 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:21.552+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:21.596 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:21.589+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:21.736 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:21.727+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:21.863 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:21.856+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:21.873 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:21.863+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:21.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:21.905+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:22.071 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:22.061+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:22.593 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:22.582+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:22.600 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:22.594+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:22.738 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:22.728+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:22.835 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:22.825+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:22.842 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:22.835+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:22.888 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:22.881+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:23.059 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:23.050+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:23.593 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:23.584+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:23.651 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:23.644+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:23.722 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:23.713+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:23.809 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:23.803+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:23.829 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:23.820+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:23.884 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:23.879+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:24.109 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:24.100+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:24.558 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:24.548+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:24.630 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:24.623+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:24.772 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:24.764+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:24.843 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:24.834+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:24.855 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:24.849+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:24.899 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:24.893+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:25.069 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:25.060+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:25.573 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:25.564+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:25.677 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:25.671+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:25.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:25.799+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:25.814 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:25.807+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:25.865 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:25.856+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:25.880 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:25.875+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:26.048 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:26.039+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:26.536 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:26.527+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:26.703 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:26.697+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:26.781 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:26.772+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:26.847 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:26.840+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:26.856 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:26.850+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:26.868 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:26.859+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:27.085 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:27.076+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:27.528 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:27.519+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:27.689 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:27.683+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:27.757 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:27.749+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:27.821 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:27.814+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:27.861 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:27.855+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:27.892 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:27.884+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:28.091 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:28.083+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:28.487 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:28.478+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:28.709 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:28.704+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:28.754 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:28.745+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:28.795 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:28.787+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:28.890 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:28.881+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:28.896 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:28.890+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:29.094 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:29.086+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:29.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:29.444+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:29.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:29.664+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:29.763 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:29.755+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:29.797 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:29.791+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:29.905 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:29.901+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:29.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:29.924+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:30.155 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:30.141+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:30.485 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:30.476+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:30.650 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:30.639+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:30.805 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:30.798+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:30.807 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:30.802+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:30.932 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:30.927+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:30.933 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:30.926+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:31.209 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:31.192+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:31.520 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:31.512+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:31.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:31.597+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:31.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:31.778+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:31.812 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:31.806+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:31.901 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:31.896+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:31.914 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:31.906+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:32.172 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:32.164+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:32.536 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:32.527+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:32.635 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:32.629+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:32.774 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:32.768+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:32.819 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:32.810+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:32.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:32.859+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:32.876 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:32.869+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:33.132 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:33.125+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:33.550 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:33.543+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:33.653 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:33.648+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:33.728 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:33.723+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:33.797 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:33.783+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:33.837 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:33.826+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:33.842 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:33.836+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:34.168 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:34.159+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:34.511 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:34.502+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:34.654 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:34.647+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:34.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:34.746+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:34.797 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:34.789+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:34.831 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:34.825+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:34.876 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:34.869+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:35.136 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:35.128+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:35.564 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:35.556+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:35.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:35.664+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:35.709 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:35.702+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:35.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:35.814+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:35.881 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:35.871+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:35.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:35.890+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:36.099 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:36.091+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:36.521 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:36.513+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:36.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:36.662+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:36.676 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:36.670+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:36.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:36.814+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:36.904 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:36.898+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:36.920 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:36.911+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:37.105 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:37.096+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:37.529 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:37.520+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:37.634 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:37.628+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:37.695 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:37.687+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:37.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:37.817+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:37.881 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:37.873+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:37.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:37.944+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:38.106 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:38.089+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:38.573 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:38.563+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:38.661 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:38.655+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:38.681 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:38.675+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:38.843 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:38.835+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:38.848 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:38.840+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:38.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:38.968+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:39.110 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:39.102+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:39.597 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:39.584+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:39.626 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:39.620+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:39.659 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:39.650+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:39.822 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:39.814+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:39.827 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:39.819+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:40.005 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:39.999+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:40.094 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:40.084+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:40.585 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:40.578+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:40.612 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:40.604+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:40.681 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:40.676+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:40.786 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:40.778+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:40.856 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:40.848+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:40.969 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:40.963+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:41.102 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:41.093+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:41.618 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:41.612+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:41.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:41.647+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:41.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:41.677+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:41.819 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:41.811+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:41.825 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:41.817+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:41.971 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:41.966+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:42.101 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:42.093+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:42.625 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:42.617+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:42.641 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:42.636+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:42.686 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:42.681+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:42.801 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:42.792+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:42.857 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:42.848+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:42.995 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:42.988+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:43.092 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:43.083+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:43.583 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:43.574+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:43.676 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:43.669+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:43.688 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:43.679+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:43.761 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:43.753+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:43.858 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:43.850+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:43.955 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:43.948+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:44.085 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:44.078+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:44.588 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:44.578+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:44.684 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:44.679+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:44.709 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:44.691+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:44.744 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:44.736+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:44.906 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:44.898+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:44.924 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:44.917+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:45.072 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:45.064+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:45.576 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:45.568+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:45.672 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:45.665+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:45.713 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:45.707+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:45.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:45.729+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:45.896 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:45.887+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:45.971 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:45.966+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:46.033 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:46.026+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:46.604 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:46.596+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:46.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:46.664+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:46.696 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:46.688+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:46.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:46.717+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:46.913 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:46.905+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:46.964 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:46.957+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:47.026 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:47.018+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:47.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:47.599+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:47.626 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:47.618+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:47.695 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:47.687+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:47.764 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:47.757+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:47.941 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:47.933+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:48.002 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:47.995+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:48.025 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:06:48.019+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e409 v409) 2019-11-29T18:06:48.037 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:48.030+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:48.593 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:48.585+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:48.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:48.595+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:48.740 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:48.732+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:48.789 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:48.783+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:48.902 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:48.894+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:49.024 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:49.016+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:49.052 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:49.045+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:49.624 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:49.617+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:49.627 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:49.619+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:49.740 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:49.731+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:49.770 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:49.763+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:49.894 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:49.886+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:49.995 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:49.988+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:50.015 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:50.009+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:50.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:50.647+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:50.667 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:50.660+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:50.740 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:50.732+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:50.760 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:50.751+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:50.854 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:50.846+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:51.031 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:51.024+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:51.049 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:51.042+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:51.692 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:51.686+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:51.698 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:51.687+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:51.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:51.740+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:51.776 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:51.763+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:51.809 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:51.801+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:52.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:52.031+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:52.047 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:52.040+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:52.659 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:06:52.651+0000 1694c700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.264720+0000 front 2019-11-29T18:05:37.748738+0000 (oldest deadline 2019-11-29T18:06:20.164709+0000) 2019-11-29T18:06:52.730 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:06:52.721+0000 1694c700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.036052+0000 front 2019-11-29T18:05:36.060031+0000 (oldest deadline 2019-11-29T18:06:20.677844+0000) 2019-11-29T18:06:52.781 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:06:52.773+0000 1694c700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.697063+0000 front 2019-11-29T18:05:36.096137+0000 (oldest deadline 2019-11-29T18:06:21.373774+0000) 2019-11-29T18:06:52.793 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:06:52.785+0000 1694c700 -1 osd.6 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:32.722236+0000 front 2019-11-29T18:05:31.824503+0000 (oldest deadline 2019-11-29T18:06:17.585832+0000) 2019-11-29T18:06:52.831 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:06:52.824+0000 1694c700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:36.812955+0000 front 2019-11-29T18:05:36.869515+0000 (oldest deadline 2019-11-29T18:06:20.290399+0000) 2019-11-29T18:06:53.032 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:06:53.022+0000 1694c700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.039923+0000 front 2019-11-29T18:05:35.023602+0000 (oldest deadline 2019-11-29T18:06:19.658639+0000) 2019-11-29T18:06:53.066 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:06:53.058+0000 1694c700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:05:35.087396+0000 front 2019-11-29T18:05:36.034790+0000 (oldest deadline 2019-11-29T18:06:20.917041+0000) 2019-11-29T18:06:53.097 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:06:53.092+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e409 v409) 2019-11-29T18:06:58.349 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:11.237 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.396 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:11.284 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.400 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:11.288 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.402 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:11.289 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.464 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:11.236 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.471 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:11.243 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.568 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:11.455 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.678 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:11.566 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:06:58.779 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:11.551 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.469 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:21.354 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.471 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:21.359 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.485 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:21.372 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.493 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:21.380 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.495 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:21.268 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.501 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:21.273 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.518 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:54:21.406 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.537 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:54:21.424 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:08.593 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:54:21.365 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:07:50.322 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:50.314+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:50.852 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:50.845+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:51.349 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:51.341+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:51.896 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:51.891+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:52.382 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:52.375+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:52.387 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:52.379+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:52.870 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:52.865+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:53.353 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:07:53.269+0000 20e22700 -1 RGWWatcher::handle_error cookie 823002512 err (107) Transport endpoint is not connected 2019-11-29T18:07:53.371 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:53.363+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:53.427 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:53.418+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:53.778 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:53.770+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:53.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:53.878+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:54.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:54.411+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:54.464 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:54.455+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:54.537 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:54.531+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:54.812 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:54.804+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:54.854 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:54.848+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:55.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:55.409+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:55.506 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:55.498+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:55.529 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:55.523+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:55.810 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:55.802+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:55.828 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:55.824+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:55.980 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:07:55.972+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:07:56.374 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:56.366+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:56.550 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:56.541+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:56.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:56.546+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:56.800 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:56.792+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:56.862 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:56.856+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:56.986 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:07:56.978+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:07:57.042 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:07:57.038+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:07:57.413 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:57.405+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:57.533 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:57.524+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:57.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:57.594+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:57.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:57.758+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:57.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:57.876+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:57.972 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:07:57.964+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:07:58.091 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:07:58.085+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:07:58.427 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:58.419+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:58.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:58.554+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:58.571 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:58.563+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:58.772 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:58.764+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:58.877 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:58.871+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:59.014 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:07:59.006+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:07:59.073 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:07:59.065+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:07:59.460 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:07:59.452+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:07:59.576 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:07:59.570+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:07:59.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:07:59.616+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:07:59.791 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:07:59.784+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:07:59.889 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:07:59.884+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:07:59.977 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:07:59.970+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:00.061 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:00.056+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:00.486 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:00.476+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:00.550 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:00.544+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:00.657 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:00.649+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:00.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:00.834+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:00.890 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:00.884+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:00.958 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:00.950+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:01.090 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:01.084+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:01.496 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:01.488+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:01.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:01.547+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:01.668 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:01.660+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:01.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:01.879+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:01.889 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:01.882+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:01.932 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:01.925+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:02.139 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:02.134+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:02.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:02.518+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:02.576 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:02.570+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:02.647 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:02.639+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:02.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:02.888+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:02.896 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:02.881+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:02.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:02.906+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:03.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:03.141+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:03.547 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:03.542+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:03.557 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:03.550+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:03.602 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:03.595+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:03.893 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:03.887+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:03.933 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:03.923+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:03.933 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:03.924+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:04.154 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:04.148+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:04.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:04.153+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:04.518 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:04.510+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:04.592 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:04.587+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:04.623 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:04.616+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:04.891 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:04.886+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:04.919 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:04.911+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:04.974 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:04.966+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:05.188 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:05.182+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:05.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:05.186+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:05.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:05.505+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:05.611 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:05.606+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:05.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:05.629+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:05.768 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:08:05.745+0000 20e22700 -1 RGWWatcher::handle_error cookie 5487098752 err (107) Transport endpoint is not connected 2019-11-29T18:08:05.873 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:05.867+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:05.911 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:05.904+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:06.012 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:06.005+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:06.164 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:06.158+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:06.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:06.160+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:06.563 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:06.555+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:06.606 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:06.598+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:06.630 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:06.622+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:06.869 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:06.863+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:06.946 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:06.938+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:06.977 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:06.970+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:07.169 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:07.161+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:07.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:07.165+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:07.607 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:07.599+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:07.610 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:07.604+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:07.656 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:07.648+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:07.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:07.895+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:07.974 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:07.967+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:07.990 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:07.982+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:08.185 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:08.179+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:08.186 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:08.181+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:08.622 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:08.614+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:08.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:08.630+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:08.652 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:08.646+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:08.899 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:08.892+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:08.956 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:08.948+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:08.967 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:08.959+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:09.220 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:09.208+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:09.222 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:09.212+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:09.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:09.576+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:09.616 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:09.610+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:09.617 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:09.609+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:09.899 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:09.892+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:09.918 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:09.910+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:09.996 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:09.986+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:10.271 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:10.263+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:10.272 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:10.266+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:10.573 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:10.564+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:10.599 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:10.593+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:10.600 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:10.595+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:10.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:10.629+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:10.924 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:10.917+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:10.925 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:08:10.917+0000 1694c700 -1 osd.6 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.923781+0000 front 2019-11-29T18:07:09.924873+0000 (oldest deadline 2019-11-29T18:07:55.197714+0000) 2019-11-29T18:08:11.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:11.032+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:11.313 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:11.308+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:11.314 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:11.309+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:11.539 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:11.531+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:11.621 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:11.614+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:11.621 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:11.616+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:11.629 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:11.622+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:11.960 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:11.954+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:12.094 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:12.084+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:12.297 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:12.291+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:12.300 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:12.295+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:12.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:12.528+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:12.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:12.597+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:12.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:12.599+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:12.614 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:12.606+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:12.941 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:12.934+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:13.058 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:13.051+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:13.267 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:13.259+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:13.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:13.262+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:13.554 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:13.546+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:13.582 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:13.577+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:13.583 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:13.576+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:13.583 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:13.578+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:13.946 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:13.940+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:14.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:14.020+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:14.287 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:14.280+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:14.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:14.285+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:14.548 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:14.540+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:14.553 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:14.545+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:14.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:14.548+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:14.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:14.575+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:14.934 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:14.929+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:15.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:15.028+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:15.318 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:15.312+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:15.319 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:15.314+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:15.584 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:15.575+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:15.588 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:15.582+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:15.589 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:15.584+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:15.633 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:15.624+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:15.956 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:15.951+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:16.060 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:16.052+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:16.329 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:16.323+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:16.333 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:16.327+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:16.564 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:16.555+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:16.605 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:16.597+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:16.634 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:16.628+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:16.635 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:16.630+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:17.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:17.004+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:17.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:17.026+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:17.331 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:17.323+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:17.331 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:17.326+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:17.571 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:17.528+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:17.598 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:17.589+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:17.649 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:17.643+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:17.650 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:17.644+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:18.047 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:18.039+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:18.076 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:18.067+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:18.302 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:18.291+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:18.302 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:18.294+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:18.505 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:18.495+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:18.593 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:18.582+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:18.632 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:18.627+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:18.634 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:18.629+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:19.034 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:19.028+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:19.103 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:19.094+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:19.264 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:19.255+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:19.266 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:19.259+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:19.490 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:19.481+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:19.639 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:19.632+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:19.641 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:19.636+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:19.643 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:19.635+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:20.026 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:20.020+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:20.119 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:20.110+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:20.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:20.222+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:20.232 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:20.225+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:20.460 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:20.451+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:20.619 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:20.612+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:20.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:20.614+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:20.684 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:20.676+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:21.034 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:21.028+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:21.090 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:21.082+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:21.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:21.233+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:21.243 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:21.237+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:21.430 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:21.422+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:21.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:21.620+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:21.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:21.624+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:21.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:21.640+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:22.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:21.999+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:22.084 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:22.075+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:22.246 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:22.237+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:22.247 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:22.240+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:22.402 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:22.392+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:22.660 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:22.651+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:22.667 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:22.660+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:22.668 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:22.662+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:23.017 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:23.010+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:23.117 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:23.106+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:23.208 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:23.199+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:23.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:23.203+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:23.450 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:23.442+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:23.516 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:08:23.507+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e415 v415) 2019-11-29T18:08:23.646 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:23.639+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:23.648 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:23.643+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:23.691 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:23.684+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:24.005 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:23.997+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:24.126 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:24.108+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:24.163 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:24.155+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:24.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:24.158+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:24.436 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:24.427+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:24.663 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:24.657+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:24.665 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:24.659+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:24.745 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:24.737+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:24.976 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:24.969+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:25.093 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:25.085+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:25.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:25.141+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:25.154 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:25.146+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:25.416 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:25.409+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:25.700 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:25.690+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:25.700 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:25.693+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:25.760 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:25.753+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:25.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:25.960+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:26.106 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:26.097+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:26.150 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:26.143+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:26.154 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:26.147+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:26.401 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:26.387+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:26.738 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:26.731+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:26.746 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:26.739+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:26.750 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:26.744+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:26.968 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:26.961+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:27.139 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:27.132+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:27.144 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:27.136+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:27.158 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:27.150+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:27.631 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:27.422+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:27.696 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:27.689+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:27.734 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:27.727+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:27.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:27.728+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:27.941 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:27.935+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:28.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:28.133+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:28.149 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:28.137+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:28.205 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:28.197+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:28.433 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:28.425+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:28.535 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:08:28.528+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e415 v415) 2019-11-29T18:08:28.662 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:28.654+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:28.737 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:28.728+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:28.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:28.730+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:28.899 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:28.891+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:29.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:29.091+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:29.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:29.094+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:29.204 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:29.197+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:29.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:29.453+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:29.660 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:29.652+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:29.767 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:29.760+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:29.777 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:29.765+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:29.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:29.876+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:30.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:30.109+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:30.118 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:30.112+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:30.203 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:30.196+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:30.497 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:30.488+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:30.653 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:30.645+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:30.765 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:30.758+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:30.766 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:30.760+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:30.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:30.887+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:31.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:31.091+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:31.102 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:31.095+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:31.173 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:31.161+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:31.477 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:31.468+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:31.649 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:31.642+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:31.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:31.766+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:31.778 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:31.768+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:31.874 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:31.867+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:32.111 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:32.102+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:32.112 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:32.106+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:32.141 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:32.133+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:32.501 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:32.490+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:32.656 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:32.648+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:32.780 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:32.774+0000 1694c700 -1 osd.3 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.406753+0000 front 2019-11-29T18:07:09.420599+0000 (oldest deadline 2019-11-29T18:07:54.106856+0000) 2019-11-29T18:08:32.783 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:32.777+0000 1694c700 -1 osd.3 415 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:32.864 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:08:32.857+0000 1694c700 -1 osd.2 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:05.944861+0000 front 2019-11-29T18:07:06.006069+0000 (oldest deadline 2019-11-29T18:07:50.606166+0000) 2019-11-29T18:08:33.142 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:08:33.134+0000 1694c700 -1 osd.5 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:09.028606+0000 front 2019-11-29T18:07:09.010534+0000 (oldest deadline 2019-11-29T18:07:53.026424+0000) 2019-11-29T18:08:33.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:33.140+0000 1694c700 -1 osd.0 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:07:23.657872+0000 (oldest deadline 2019-11-29T18:08:03.657872+0000) 2019-11-29T18:08:33.150 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:08:33.143+0000 1694c700 -1 osd.0 415 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e413) 2019-11-29T18:08:33.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:08:33.484+0000 1694c700 -1 osd.7 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:07.731322+0000 front 2019-11-29T18:07:07.569131+0000 (oldest deadline 2019-11-29T18:07:51.612890+0000) 2019-11-29T18:08:33.587 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:08:33.575+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e415 v415) 2019-11-29T18:08:33.630 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:08:33.622+0000 1694c700 -1 osd.4 415 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:07:04.376191+0000 front 2019-11-29T18:07:04.384508+0000 (oldest deadline 2019-11-29T18:07:50.266674+0000) 2019-11-29T18:08:33.963 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:33.898+0000 1694c700 -1 osd.3 416 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:08:34.925 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:08:34.906+0000 1694c700 -1 osd.3 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:108721 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5487098752] snapc 0=[] ondisk+write+known_if_redirected e415) 2019-11-29T18:09:34.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:09:34.417+0000 1694c700 -1 osd.0 420 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108439 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected e419) 2019-11-29T18:09:35.642 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:09:35.498+0000 1694c700 -1 osd.0 420 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:108439 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected e419) 2019-11-29T18:10:26.418 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:26.405+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:26.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:26.637+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:27.438 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:27.430+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:27.664 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:27.656+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:27.681 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:27.673+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:27.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:27.681+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:27.944 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:27.936+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:28.453 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:28.445+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:28.676 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:28.669+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:28.677 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:28.671+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:28.694 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:28.686+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:28.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:28.914+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:29.420 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:29.412+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:29.441 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:29.434+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:29.647 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:29.639+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:29.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:29.664+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:29.674 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:29.666+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:29.777 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:29.769+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:29.954 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:29.946+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:30.422 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:30.415+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:30.469 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:30.462+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:30.653 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:30.646+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:30.655 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:30.647+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:30.680 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:30.673+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:30.774 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:30.768+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:31.002 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:30.994+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:31.414 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:31.407+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:31.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:31.467+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:31.643 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:31.635+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:31.687 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:31.681+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:31.691 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:31.683+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:31.736 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:31.730+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:32.019 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:32.010+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:32.425 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:32.417+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:32.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:32.447+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:32.663 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:32.655+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:32.700 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:32.691+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:32.704 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:32.697+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:32.730 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:32.722+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:33.040 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:33.030+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:33.400 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:33.392+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:33.427 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:33.421+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:33.698 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:33.691+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:33.701 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:33.695+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:33.716 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:33.711+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:33.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:33.739+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:34.044 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:34.034+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:34.428 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:34.418+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:34.452 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:34.442+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:34.711 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:34.700+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:34.727 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:34.720+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:34.729 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:34.721+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:34.798 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:34.791+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:35.061 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:35.052+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:35.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:35.447+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:35.480 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:35.472+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:35.735 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:35.728+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:35.749 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:35.741+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:35.753 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:35.746+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:35.833 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:35.821+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:36.030 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:36.022+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:36.493 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:36.486+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:36.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:36.488+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:36.730 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:36.724+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:36.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:36.745+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:36.775 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:36.767+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:36.787 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:36.779+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:36.992 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:36.981+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:37.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:37.460+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:37.507 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:37.499+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:37.711 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:37.705+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:37.745 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:37.738+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:37.759 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:37.751+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:37.830 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:37.822+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:38.030 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:38.022+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:38.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:38.483+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:38.537 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:38.529+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:38.670 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:38.663+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:38.797 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:38.789+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:38.804 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:38.797+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:38.812 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:38.804+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:39.055 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:39.048+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:39.479 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:39.472+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:39.532 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:39.520+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:39.677 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:39.670+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:39.773 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:39.766+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:39.796 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:39.789+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:39.832 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:39.824+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:40.100 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:40.092+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:40.508 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:40.502+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:40.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:40.508+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:40.702 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:40.634+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:40.754 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:40.748+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:40.763 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:40.755+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:40.812 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:40.805+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:41.127 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:41.120+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:41.506 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:41.496+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:41.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:41.546+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:41.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:41.593+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:41.758 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:41.750+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:41.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:41.784+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:41.868 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:41.859+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:42.158 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:42.150+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:42.550 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:42.540+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:42.565 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:42.559+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:42.593 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:42.586+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:42.794 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:42.786+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:42.798 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:42.790+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:42.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:42.878+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:43.203 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:43.195+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:43.526 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:43.518+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:43.543 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:43.535+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:43.623 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:43.614+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:43.771 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:43.764+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:43.776 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:43.768+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:43.850 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:43.842+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:44.258 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:44.235+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:44.505 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:44.498+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:44.533 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:44.525+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:44.669 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:44.663+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:44.746 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:44.739+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:44.814 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:44.806+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:44.854 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:44.846+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:45.237 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:45.227+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:45.529 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:45.521+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:45.541 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:45.533+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:45.709 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:45.701+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:45.747 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:45.723+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:45.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:45.830+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:45.889 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:45.881+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:46.273 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:46.263+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:46.506 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:46.496+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:46.527 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:46.519+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:46.697 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:46.689+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:46.700 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:46.693+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:46.883 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:46.870+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:46.945 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:46.935+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:47.285 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:47.277+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:47.493 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:47.486+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:47.534 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:47.525+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:47.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:47.666+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:47.689 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:47.682+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:47.915 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:47.906+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:47.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:47.956+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:48.301 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:48.292+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:48.517 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:48.510+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:48.553 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:48.545+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:48.658 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:48.652+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:48.715 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:48.707+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:48.939 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:48.929+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:49.001 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:48.992+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:49.351 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:49.342+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:49.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:49.515+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:49.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:49.606+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:49.689 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:49.683+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:49.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:49.728+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:49.899 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:49.890+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:50.045 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:50.036+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:50.350 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:50.341+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:50.534 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:50.527+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:50.663 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:50.654+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:50.701 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:50.693+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:50.705 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:50.695+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:50.948 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:50.939+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:51.048 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:51.039+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:51.339 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:51.330+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:51.585 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:51.579+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:51.648 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:51.639+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:51.716 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:51.709+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:51.750 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:51.743+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:51.993 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:51.984+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:52.060 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:52.051+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:52.351 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:52.342+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:52.556 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:52.549+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:52.651 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:52.643+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:52.734 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:52.726+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:52.748 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:52.741+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:52.953 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:52.945+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:53.092 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:53.083+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:53.313 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:53.304+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:53.564 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:53.557+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:53.630 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:53.621+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:53.688 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:53.683+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:53.775 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:53.769+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:53.973 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:53.964+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:54.085 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:54.076+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:54.321 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:54.313+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:54.576 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:54.570+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:54.633 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:54.624+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:54.688 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:54.683+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:54.739 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:54.733+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:54.966 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:54.958+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:55.087 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:55.078+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:55.310 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:55.301+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:55.578 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:55.572+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:55.634 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:55.624+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:55.692 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:55.685+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:55.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:55.727+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:55.944 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:55.936+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:56.113 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:56.105+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:56.317 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:56.310+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:56.590 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:56.584+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:56.651 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:56.643+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:56.673 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:56.668+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:56.723 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:56.718+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:56.994 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:56.986+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:57.103 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:57.097+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:57.299 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:57.291+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:57.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:57.625+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:57.638 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:57.631+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:57.664 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:57.657+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:57.734 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:57.728+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:57.957 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:57.948+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:58.100 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:58.092+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:10:58.310 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:10:58.302+0000 1694c700 -1 osd.4 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.689118+0000 front 2019-11-29T18:09:43.952555+0000 (oldest deadline 2019-11-29T18:10:27.664705+0000) 2019-11-29T18:10:58.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:10:58.625+0000 1694c700 -1 osd.2 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.857985+0000 front 2019-11-29T18:09:41.897728+0000 (oldest deadline 2019-11-29T18:10:26.865153+0000) 2019-11-29T18:10:58.633 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:10:58.626+0000 1694c700 -1 osd.1 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.370614+0000 front 2019-11-29T18:09:44.599222+0000 (oldest deadline 2019-11-29T18:10:29.056900+0000) 2019-11-29T18:10:58.678 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:10:58.672+0000 1694c700 -1 osd.7 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.453409+0000 front 2019-11-29T18:09:44.327332+0000 (oldest deadline 2019-11-29T18:10:25.859701+0000) 2019-11-29T18:10:58.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:10:58.749+0000 1694c700 -1 osd.3 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:43.543720+0000 front 2019-11-29T18:09:43.659581+0000 (oldest deadline 2019-11-29T18:10:29.370480+0000) 2019-11-29T18:10:58.959 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:10:58.951+0000 1694c700 -1 osd.6 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:41.781557+0000 front 2019-11-29T18:09:41.785231+0000 (oldest deadline 2019-11-29T18:10:27.440913+0000) 2019-11-29T18:10:59.070 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:10:59.063+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e420 v420) 2019-11-29T18:10:59.090 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:10:59.083+0000 1694c700 -1 osd.5 420 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:09:44.324857+0000 front 2019-11-29T18:09:44.326019+0000 (oldest deadline 2019-11-29T18:10:26.346269+0000) 2019-11-29T18:11:08.102 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:58:20.990 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.110 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:58:20.998 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.123 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:58:21.010 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.132 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:58:20.904 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.132 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:58:21.020 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.155 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:58:20.927 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.371 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:58:21.259 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.416 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:58:21.302 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:08.451 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:58:21.224 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:11:35.741 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:11:35.638+0000 34271700 -1 osd.6 424 build_incremental_map_msg missing incremental map 424 2019-11-29T18:11:35.890 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:11:35.865+0000 37277700 -1 osd.6 424 build_incremental_map_msg missing incremental map 424 2019-11-29T18:11:36.821 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:11:36.812+0000 114ab700 -1 osd.6 425 build_incremental_map_msg missing incremental map 424 2019-11-29T18:11:56.479 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:11:56.471+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:11:57.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:11:57.206+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:11:57.492 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:11:57.484+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:11:57.815 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:11:57.807+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:11:58.209 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:11:58.200+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:11:58.366 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:11:58.343+0000 20e22700 -1 RGWWatcher::handle_error cookie 384252496 err (107) Transport endpoint is not connected 2019-11-29T18:11:58.474 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:11:58.467+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:11:58.725 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:11:58.718+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:11:58.858 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:11:58.850+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:11:59.119 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:11:59.112+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:11:59.199 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:11:59.193+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:11:59.467 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:11:59.459+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:11:59.759 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:11:59.752+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:11:59.840 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:11:59.833+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:00.151 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:00.142+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:00.172 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:00.166+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:00.456 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:00.448+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:00.738 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:00.733+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:00.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:00.863+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:01.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:01.112+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:01.131 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:01.125+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:01.493 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:01.485+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:01.757 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:01.740+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:01.888 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:01.880+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:02.143 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:02.138+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:02.167 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:02.159+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:02.526 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:02.517+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:02.717 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:02.711+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:02.933 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:02.924+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:03.175 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:03.170+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:03.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:03.180+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:03.567 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:03.558+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:03.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:03.719+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:03.953 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:03.944+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:04.192 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:04.182+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:04.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:04.187+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:04.607 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:04.598+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:04.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:04.747+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:04.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:04.931+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:05.217 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:05.208+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:05.228 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:05.222+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:05.653 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:05.644+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:05.741 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:05.727+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:05.915 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:05.906+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:06.184 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:06.175+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:06.252 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:06.245+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:06.615 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:06.607+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:06.747 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:06.738+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:06.749 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:06.742+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:06.964 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:06.955+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:07.239 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:07.229+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:07.271 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:07.264+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:07.582 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:07.575+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:07.612 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:07.603+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:07.746 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:07.737+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:07.768 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:07.761+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:07.973 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:07.964+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:08.265 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:08.253+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:08.284 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:08.278+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:08.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:08.568+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:08.595 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:08.587+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:08.767 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:08.759+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:08.793 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:08.787+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:08.971 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:08.959+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:09.249 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:09.234+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:09.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:09.250+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:09.536 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:09.529+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:09.596 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:09.587+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:09.770 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:09.761+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:09.817 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:09.810+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:10.019 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:10.009+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:10.265 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:10.259+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:10.270 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:10.262+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:10.568 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:10.562+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:10.650 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:10.641+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:10.789 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:10.780+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:10.807 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:10.799+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:10.986 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:10.971+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:11.293 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:11.286+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:11.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:11.315+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:11.561 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:11.555+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:11.608 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:11.599+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:11.788 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:11.780+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:11.845 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:11.839+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:11.942 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:11.932+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:12.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:12.293+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:12.319 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:12.312+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:12.559 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:12.553+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:12.655 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:12.647+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:12.762 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:12.755+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:12.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:12.799+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:12.937 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:12.929+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:13.305 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:13.297+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:13.323 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:13.316+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:13.533 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:13.520+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:13.533 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:13.523+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:13.696 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:13.685+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:13.775 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:13.767+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:13.860 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:13.852+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:13.927 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:13.920+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:14.279 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:14.272+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:14.318 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:14.311+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:14.403 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:12:14.379+0000 20e22700 -1 RGWWatcher::handle_error cookie 364116080 err (107) Transport endpoint is not connected 2019-11-29T18:12:14.520 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:14.511+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:14.521 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:14.515+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:14.700 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:14.693+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:14.817 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:14.807+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:14.827 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:14.816+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:14.972 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:14.964+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:15.285 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:15.277+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:15.311 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:15.305+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:15.570 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:15.562+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:15.571 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:15.565+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:15.715 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:15.708+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:15.804 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:15.797+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:15.842 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:15.834+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:15.973 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:15.965+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:16.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:16.259+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:16.309 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:16.302+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:16.523 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:16.516+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:16.523 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:16.517+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:16.703 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:16.695+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:16.760 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:16.753+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:16.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:16.801+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:17.021 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:17.014+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:17.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:17.257+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:17.266 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:17.260+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:17.334 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:17.326+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:17.500 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:17.492+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:17.502 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:17.495+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:17.668 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:17.658+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:17.792 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:17.786+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:17.816 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:17.809+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:17.996 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:17.988+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:18.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:18.303+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:18.313 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:18.305+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:18.328 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:18.320+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:18.536 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:18.528+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:18.541 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:18.530+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:18.711 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:18.703+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:18.774 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:18.767+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:18.853 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:18.844+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:19.018 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:19.010+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:19.301 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:19.293+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:19.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:19.298+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:19.331 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:19.323+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:19.497 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:59:32.385 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.514 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:59:32.402 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.533 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:59:32.405 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.540 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:19.534+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:19.541 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:19.536+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:19.553 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:59:32.431 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.578 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:59:32.351 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.584 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:59:32.356 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.615 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:06:59:32.501 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.657 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:06:59:32.535 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.717 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:06:59:32.490 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:12:19.742 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:19.734+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:19.793 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:19.785+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:19.836 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:19.829+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:20.030 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:20.018+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:20.293 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:20.287+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:20.294 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:20.288+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:20.349 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:20.338+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:20.587 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:20.579+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:20.588 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:20.580+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:20.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:20.735+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:20.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:20.762+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:20.865 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:20.857+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:21.030 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:21.023+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:21.316 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:21.309+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:21.318 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:21.312+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:21.354 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:21.347+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:21.628 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:21.622+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:21.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:21.625+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:21.756 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:21.748+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:21.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:21.756+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:21.918 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:21.910+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:22.042 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:22.034+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:22.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:22.355+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:22.366 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:22.360+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:22.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:22.378+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:22.621 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:22.612+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:22.622 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:22.615+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:22.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:22.763+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:22.782 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:22.773+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:22.878 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:22.870+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:23.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:23.057+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:23.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:23.394+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:23.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:23.397+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:23.415 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:23.398+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:23.629 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:23.622+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:23.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:23.624+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:23.758 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:23.750+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:23.795 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:23.787+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:23.916 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:23.908+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:24.036 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:24.029+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:24.379 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:24.371+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:24.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:24.397+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:24.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:24.400+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:24.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:24.604+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:24.610 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:24.605+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:24.783 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:24.776+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:24.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:24.801+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:24.957 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:24.950+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:25.022 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:25.015+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:25.377 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:25.369+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:25.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:25.437+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:25.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:25.438+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:25.610 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:25.604+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:25.611 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:25.606+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:25.763 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:25.755+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:25.833 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:25.827+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:25.946 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:25.939+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:26.000 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:25.992+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:26.388 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:26.380+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:26.420 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:26.414+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:26.420 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:26.415+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:26.586 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:26.569+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:26.586 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:26.574+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:26.782 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:26.775+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:26.794 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:26.789+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:26.945 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:26.937+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:27.054 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:27.036+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:27.384 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:27.359+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:27.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:27.375+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:27.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:27.377+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:27.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:27.585+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:27.596 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:27.589+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:27.765 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:27.758+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:27.794 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:27.786+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:27.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:27.941+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:28.075 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:28.068+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:28.353 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:28.339+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:28.353 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:28.341+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:28.365 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:28.357+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:28.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:28.627+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:28.641 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:28.636+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:28.759 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:28.753+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:28.848 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:28.840+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:28.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:28.941+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:29.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:29.109+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:29.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:29.296+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:29.308 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:29.297+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:29.366 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:29.359+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:29.477 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:12:29.471+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e425 v425) 2019-11-29T18:12:29.620 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:29.614+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:29.621 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:29.616+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:29.801 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:29.796+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:29.836 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:29.829+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:29.915 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:29.908+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:30.143 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:30.135+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:30.327 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:30.320+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:30.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:30.322+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:30.376 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:30.369+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:30.623 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:30.618+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:30.627 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:30.622+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:30.835 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:30.829+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:30.839 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:30.832+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:30.940 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:30.932+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:31.153 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:31.137+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:31.326 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:31.319+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:31.327 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:31.322+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:31.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:31.391+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:31.665 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:31.660+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:31.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:31.662+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:31.827 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:31.820+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:31.863 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:31.857+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:31.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:31.967+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:32.132 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:32.122+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:32.340 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:32.335+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:32.344 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:32.339+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:32.411 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:32.403+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:32.706 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:32.698+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:32.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:32.702+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:32.861 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:32.850+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:32.902 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:32.897+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:32.995 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:32.987+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:33.184 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:33.175+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:33.371 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:33.364+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:33.394 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:33.389+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:33.396 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:33.392+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:33.677 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:33.671+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:33.678 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:33.673+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:33.831 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:33.823+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:33.906 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:33.897+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:34.040 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:34.032+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:34.233 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:34.224+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:34.389 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:34.382+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:34.433 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:34.427+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:34.435 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:34.431+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:34.496 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:12:34.489+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e425 v425) 2019-11-29T18:12:34.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:34.690+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:34.699 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:34.693+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:34.820 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:34.812+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:34.944 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:34.938+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:35.060 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:35.052+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:35.199 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:35.192+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:35.365 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:35.358+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:35.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:35.397+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:35.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:35.398+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:35.735 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:35.729+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:35.739 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:35.734+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:35.870 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:35.862+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:35.919 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:35.912+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:36.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:36.095+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:36.223 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:36.216+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:36.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:36.383+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:36.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:36.384+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:36.412 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:36.405+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:36.788 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:36.782+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:36.792 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:36.787+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:36.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:36.880+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:36.928 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:36.922+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:37.104 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:37.097+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:37.218 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:37.211+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:37.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:37.339+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:37.348 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:37.342+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:37.461 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:37.453+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:37.762 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:37.755+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:37.762 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:37.758+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:37.872 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:37.864+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:37.949 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:37.942+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:38.120 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:38.111+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:38.255 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:38.248+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:38.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:38.380+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:38.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:38.382+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:38.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:38.435+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:38.723 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:38.718+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:38.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:38.722+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:38.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:38.835+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:38.905 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:38.898+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:39.101 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:39.094+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:39.251 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:39.243+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:39.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:39.424+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:39.432 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:39.427+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:39.490 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:39.484+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:39.522 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:12:39.512+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e425 v425) 2019-11-29T18:12:39.752 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:39.746+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:39.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:39.748+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:39.816 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:39.809+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:39.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:39.878+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:40.066 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:40.058+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:40.264 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:40.257+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:40.446 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:40.441+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:40.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:40.443+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:40.535 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:40.525+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:40.736 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:40.730+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:40.738 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:40.732+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:40.845 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:40.827+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:40.886 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:40.881+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:41.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:41.058+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:41.257 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:41.248+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:41.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:41.487+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:41.495 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:41.490+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:41.569 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:41.561+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:41.719 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:41.711+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:41.720 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:41.714+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:41.828 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:41.820+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:41.885 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:41.866+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:42.066 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:42.057+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:42.291 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:42.283+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:42.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:42.453+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:42.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:42.455+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:42.605 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:42.597+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:42.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:42.677+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:42.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:42.680+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:42.815 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:42.806+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:42.906 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:42.900+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:43.093 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:43.084+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:43.273 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:43.265+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:43.495 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:43.488+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:43.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:43.490+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:43.641 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:43.632+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:43.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:43.678+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:43.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:43.681+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:43.866 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:12:43.857+0000 1694c700 -1 osd.5 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.474796+0000 front 2019-11-29T18:11:10.370395+0000 (oldest deadline 2019-11-29T18:11:56.233870+0000) 2019-11-29T18:12:43.931 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:12:43.926+0000 1694c700 -1 osd.2 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.100836+0000 front 2019-11-29T18:11:15.076928+0000 (oldest deadline 2019-11-29T18:11:57.923095+0000) 2019-11-29T18:12:44.051 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:12:44.042+0000 1694c700 -1 osd.6 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:14.268051+0000 front 2019-11-29T18:11:14.266141+0000 (oldest deadline 2019-11-29T18:12:05.957371+0000) 2019-11-29T18:12:44.235 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:12:44.225+0000 1694c700 -1 osd.4 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.466027+0000 front 2019-11-29T18:11:16.311985+0000 (oldest deadline 2019-11-29T18:11:57.718990+0000) 2019-11-29T18:12:44.534 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:44.527+0000 1694c700 -1 osd.3 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:10.819995+0000 front 2019-11-29T18:11:10.750082+0000 (oldest deadline 2019-11-29T18:11:56.633512+0000) 2019-11-29T18:12:44.541 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:44.532+0000 1694c700 -1 osd.3 425 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:44.565 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:12:44.560+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e425 v425) 2019-11-29T18:12:44.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:44.639+0000 1694c700 -1 osd.0 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:11:33.123343+0000 (oldest deadline 2019-11-29T18:12:13.123343+0000) 2019-11-29T18:12:44.645 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:12:44.636+0000 1694c700 -1 osd.7 425 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:11:15.176033+0000 front 2019-11-29T18:11:15.160206+0000 (oldest deadline 2019-11-29T18:11:58.575178+0000) 2019-11-29T18:12:44.648 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:12:44.643+0000 1694c700 -1 osd.0 425 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:109069 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e424) 2019-11-29T18:12:45.558 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:45.552+0000 1694c700 -1 osd.3 426 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:109091 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:12:46.723 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:12:46.702+0000 1694c700 -1 osd.3 427 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109122 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 364116080 gen 1] snapc 0=[] ondisk+write+known_if_redirected e425) 2019-11-29T18:14:30.904 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:30.896+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:31.791 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:31.783+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:31.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:31.892+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:32.174 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:32.165+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:32.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:32.679+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:32.837 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:32.830+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:32.942 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:32.935+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:33.021 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:33.014+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:33.198 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:33.190+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:33.655 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:33.647+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:33.864 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:33.854+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:33.996 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:33.988+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:34.005 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:33.996+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:34.145 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:34.137+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:34.228 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:34.220+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:34.612 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:34.605+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:34.881 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:34.867+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:34.931 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:34.923+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:34.983 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:34.972+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:35.054 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:35.046+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:35.170 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:35.162+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:35.257 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:35.248+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:35.595 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:35.588+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:35.917 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:35.908+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:35.940 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:35.929+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:35.951 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:35.942+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:36.021 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:36.013+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:36.218 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:36.206+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:36.250 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:36.241+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:36.607 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:36.600+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:36.879 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:36.870+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:36.924 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:36.914+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:37.005 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:36.996+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:37.058 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:37.052+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:37.229 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:37.220+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:37.242 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:37.236+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:37.627 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:37.619+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:37.932 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:37.923+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:37.961 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:37.953+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:37.966 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:37.957+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:38.065 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:38.058+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:38.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:38.226+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:38.271 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:38.263+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:38.614 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:38.606+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:38.899 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:38.890+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:38.996 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:38.988+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:39.004 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:38.995+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:39.088 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:39.079+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:39.243 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:39.236+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:39.269 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:39.260+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:39.617 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:39.610+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:39.892 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:39.884+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:40.043 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:40.036+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:40.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:40.042+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:40.058 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:40.050+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:40.203 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:40.195+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:40.271 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:40.262+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:40.580 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:40.570+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:40.915 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:40.904+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:41.022 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:41.013+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:41.046 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:41.040+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:41.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:41.078+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:41.210 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:41.205+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:41.280 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:41.271+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:41.625 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:41.618+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:41.941 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:41.932+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:42.008 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:41.999+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:42.013 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:42.008+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:42.073 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:42.064+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:42.237 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:42.227+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:42.318 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:42.309+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:42.609 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:42.601+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:42.897 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:42.888+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:43.003 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:42.995+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:43.030 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:43.023+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:43.055 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:43.046+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:43.192 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:43.185+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:43.324 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:43.316+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:43.627 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:43.621+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:43.961 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:43.942+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:44.024 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:44.015+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:44.031 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:44.025+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:44.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:44.042+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:44.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:44.177+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:44.337 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:44.328+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:44.591 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:44.584+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:44.970 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:44.963+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:44.988 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:44.982+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:45.005 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:44.997+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:45.084 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:45.076+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:45.165 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:45.159+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:45.362 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:45.352+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:45.596 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:45.581+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:45.950 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:45.943+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:45.987 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:45.982+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:46.001 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:45.993+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:46.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:46.044+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:46.152 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:46.146+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:46.327 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:46.320+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:46.620 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:46.613+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:46.972 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:46.965+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:46.985 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:46.979+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:46.988 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:46.980+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:47.017 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:47.009+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:47.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:47.176+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:47.306 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:47.298+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:47.651 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:47.644+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:47.962 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:47.957+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:47.996 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:47.988+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:48.020 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:48.009+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:48.025 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:48.016+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:48.207 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:48.202+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:48.272 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:48.263+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:48.675 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:48.669+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:48.964 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:48.957+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:48.988 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:48.981+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:48.989 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:48.982+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:49.065 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:49.057+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:49.203 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:49.197+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:49.264 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:49.256+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:49.702 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:49.697+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:49.997 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:49.990+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:50.002 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:49.996+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:50.033 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:50.021+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:50.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:50.083+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:50.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:50.162+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:50.285 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:50.277+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:50.713 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:50.706+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:50.968 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:50.961+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:51.025 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:51.017+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:51.034 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:51.027+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:51.114 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:51.106+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:51.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:51.207+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:51.245 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:51.238+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:51.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:51.746+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:51.921 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:51.915+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:52.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:52.017+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:52.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:52.057+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:52.145 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:52.138+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:52.218 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:52.211+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:52.255 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:52.244+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:52.709 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:52.702+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:52.948 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:52.941+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:53.000 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:52.990+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:53.036 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:53.028+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:53.151 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:53.142+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:53.235 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:53.227+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:53.255 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:53.249+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:53.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:53.678+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:53.930 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:53.924+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:54.029 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:54.021+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:54.040 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:54.033+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:54.127 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:54.119+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:54.257 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:54.249+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:54.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:54.291+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:54.727 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:54.722+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:54.897 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:54.891+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:55.057 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:55.049+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:55.058 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:55.051+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:55.086 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:55.078+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:55.254 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:55.246+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:55.321 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:55.314+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:55.756 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:55.750+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:55.869 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:55.855+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:56.063 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:56.055+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:56.066 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:56.058+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:56.077 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:56.069+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:56.230 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:56.222+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:56.327 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:56.321+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:56.758 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:56.752+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:56.907 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:56.901+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:57.031 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:57.023+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:57.075 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:57.067+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:57.080 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:57.072+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:57.248 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:57.240+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:57.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:57.327+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:57.783 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:57.776+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:57.942 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:57.936+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:58.008 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:58.000+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:58.075 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:58.066+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:58.132 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:58.124+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:58.230 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:58.222+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:58.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:58.379+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:58.785 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:58.778+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:58.907 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:58.901+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:14:59.003 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:14:58.995+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:14:59.095 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:14:59.087+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:14:59.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:14:59.097+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:14:59.251 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:14:59.243+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:14:59.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:14:59.380+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:14:59.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:14:59.800+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:14:59.889 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:14:59.884+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:00.022 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:15:00.014+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:15:00.086 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:00.079+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:00.095 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:15:00.088+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:15:00.209 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:00.201+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:00.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:00.417+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:15:00.822 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:15:00.817+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:15:00.879 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:15:00.874+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:01.054 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:01.046+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:01.071 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:15:01.063+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:15:01.092 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:15:01.084+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:15:01.212 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:01.203+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:01.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:01.419+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:15:01.801 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:15:01.796+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:15:01.865 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:15:01.860+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:02.086 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:15:02.078+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:15:02.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:02.094+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:02.116 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:15:02.108+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:15:02.202 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:02.194+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:02.433 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:02.428+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:15:02.773 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:15:02.768+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:15:02.836 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:15:02.826+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:03.068 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:03.060+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:03.088 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:15:03.080+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:15:03.089 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:15:03.082+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:15:03.256 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:03.247+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:03.393 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:03.388+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:15:03.760 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:15:03.754+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:15:03.816 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:15:03.810+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:04.074 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:04.067+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:04.137 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:15:04.129+0000 1694c700 -1 osd.4 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.862322+0000 front 2019-11-29T18:13:50.136146+0000 (oldest deadline 2019-11-29T18:14:30.936012+0000) 2019-11-29T18:15:04.137 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:15:04.128+0000 1694c700 -1 osd.7 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:47.116345+0000 front 2019-11-29T18:13:47.224581+0000 (oldest deadline 2019-11-29T18:14:30.808711+0000) 2019-11-29T18:15:04.298 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:04.282+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:04.391 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:04.386+0000 1694c700 -1 osd.3 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.575457+0000 front 2019-11-29T18:13:50.554412+0000 (oldest deadline 2019-11-29T18:14:33.371983+0000) 2019-11-29T18:15:04.743 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:15:04.736+0000 1694c700 -1 osd.2 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:51.312006+0000 front 2019-11-29T18:13:51.072284+0000 (oldest deadline 2019-11-29T18:14:31.807528+0000) 2019-11-29T18:15:04.863 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:15:04.857+0000 1694c700 -1 osd.1 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.870159+0000 front 2019-11-29T18:13:50.136532+0000 (oldest deadline 2019-11-29T18:14:32.342707+0000) 2019-11-29T18:15:04.989 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:15:04.980+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 41sec e430 v430) 2019-11-29T18:15:05.059 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:15:05.051+0000 1694c700 -1 osd.6 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:49.484614+0000 front 2019-11-29T18:13:49.508981+0000 (oldest deadline 2019-11-29T18:14:34.132972+0000) 2019-11-29T18:15:05.268 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:15:05.261+0000 1694c700 -1 osd.5 430 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:13:50.310708+0000 front 2019-11-29T18:13:50.481621+0000 (oldest deadline 2019-11-29T18:14:31.292481+0000) 2019-11-29T18:15:08.332 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:02:21.220 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.340 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:02:21.229 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.356 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:02:21.244 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.361 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:02:21.248 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.400 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:02:21.172 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.419 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:02:21.192 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.525 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:02:21.413 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.550 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:02:21.439 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:08.677 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:02:21.450 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:15:58.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:58.941+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:15:59.943 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:15:59.938+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:00.987 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:00.972+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:01.959 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:01.954+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:02.491 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:02.483+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:03.002 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:02.996+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:03.146 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:03.138+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:03.235 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:03.229+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:03.506 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:03.498+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:03.986 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:03.980+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:04.157 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:04.150+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:04.254 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:04.247+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:04.483 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:04.475+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:04.558 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:04.551+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:05.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:05.022+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:05.031 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:05.026+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:05.170 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:05.163+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:05.278 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:05.270+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:05.440 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:05.431+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:05.555 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:05.548+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:05.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:05.922+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:06.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:06.022+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:06.030 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:06.024+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:06.141 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:06.133+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:06.302 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:06.296+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:06.469 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:06.461+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:06.598 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:06.590+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:06.897 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:06.890+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:07.003 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:06.994+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:07.004 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:06.997+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:07.133 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:07.124+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:07.272 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:16:07.152+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414315072 err (107) Transport endpoint is not connected 2019-11-29T18:16:07.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:07.326+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:07.467 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:07.460+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:07.604 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:07.595+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:07.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:07.944+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:08.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:08.030+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:08.038 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:08.032+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:08.141 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:08.133+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:08.325 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:08.319+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:08.450 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:08.444+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:08.561 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:08.554+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:08.923 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:08.915+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:09.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:09.067+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:09.075 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:09.069+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:09.100 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:09.093+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:09.340 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:09.333+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:09.489 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:09.481+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:09.570 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:09.562+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:09.884 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:09.876+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:10.081 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:10.073+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:10.122 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:10.114+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:10.122 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:10.116+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:10.367 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:10.359+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:10.524 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:10.517+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:10.542 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:10.534+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:10.733 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:10.727+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:10.908 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:10.900+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:11.081 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:11.073+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:11.082 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:11.076+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:11.134 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:11.126+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:11.387 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:11.379+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:11.504 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:11.497+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:11.577 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:11.570+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:11.703 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:11.698+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:11.914 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:11.905+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:12.094 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:12.087+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:12.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:12.092+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:12.101 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:12.093+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:12.392 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:12.387+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:12.488 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:12.480+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:12.554 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:12.547+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:12.658 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:12.653+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:12.919 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:12.912+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:13.080 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:13.073+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:13.099 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:13.092+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:13.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:13.097+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:13.402 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:13.397+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:13.511 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:13.503+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:13.563 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:13.556+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:13.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:13.640+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:13.972 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:13.948+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:14.050 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:14.042+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:14.151 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:14.146+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:14.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:14.149+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:14.439 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:14.432+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:14.516 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:14.510+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:14.597 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:14.589+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:14.659 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:14.654+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:14.928 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:14.920+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:15.033 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:15.024+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:15.114 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:15.107+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:15.118 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:15.109+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:15.484 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:15.478+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:15.502 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:15.494+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:15.648 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:15.640+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:15.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:15.667+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:15.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:15.923+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:16.009 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:16.002+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:16.113 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:16.107+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:16.114 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:16.108+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:16.441 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:16.436+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:16.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:16.478+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:16.681 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:16.676+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:16.695 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:16.687+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:16.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:16.880+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:16.971 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:16.962+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:17.068 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:17.063+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:17.070 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:17.066+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:17.462 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:17.453+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:17.473 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:17.467+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:17.665 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:17.660+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:17.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:17.663+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:17.706 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:17.698+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:17.890 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:17.882+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:18.006 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:17.997+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:18.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:18.022+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:18.030 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:18.026+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:18.476 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:18.468+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:18.501 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:18.494+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:18.699 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:18.691+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:18.700 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:18.694+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:18.749 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:18.741+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:18.942 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:18.934+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:19.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:19.006+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:19.015 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:19.010+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:19.056 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:19.049+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:19.499 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:19.491+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:19.529 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:19.521+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:19.726 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:19.719+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:19.727 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:19.722+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:19.730 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:19.723+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:19.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:19.986+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:20.037 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:20.032+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:20.039 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:20.034+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:20.114 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:20.107+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:20.455 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:20.448+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:20.565 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:20.557+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:20.698 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:20.690+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:20.698 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:20.693+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:20.771 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:20.762+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:21.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:21.003+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:21.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:21.041+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:21.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:21.042+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:21.095 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:21.087+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:21.461 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:21.454+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:21.544 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:21.536+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:21.745 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:21.739+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:21.747 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:21.741+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:21.772 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:21.764+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:22.002 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:21.994+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:22.024 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:22.016+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:22.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:22.019+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:22.121 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:22.113+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:22.458 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:22.452+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:22.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:22.508+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:22.735 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:22.729+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:22.737 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:22.731+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:22.813 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:22.805+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:23.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:23.010+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:23.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:23.059+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:23.067 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:23.063+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:23.118 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:23.110+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:23.469 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:23.462+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:23.556 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:23.545+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:23.694 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:23.687+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:23.699 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:23.694+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:23.804 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:23.796+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:24.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:24.034+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:24.050 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:24.044+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:24.053 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:24.049+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:24.130 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:24.123+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:24.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:24.424+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:24.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:24.510+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:24.691 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:24.687+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:24.693 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:24.689+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:24.831 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:24.824+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:25.024 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:25.019+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:25.025 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:25.020+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:25.057 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:25.042+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:25.163 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:25.155+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:25.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:25.385+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:25.560 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:25.552+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:25.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:25.658+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:25.665 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:25.660+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:25.850 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:25.843+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:26.036 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:26.029+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:26.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:26.059+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:26.067 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:26.063+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:26.172 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:26.164+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:26.415 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:26.408+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:26.579 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:26.572+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:26.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:26.678+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:26.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:26.682+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:26.835 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:26.827+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:27.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:27.007+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:27.095 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:27.089+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:27.096 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:27.091+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:27.208 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:27.202+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:27.369 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:27.364+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:27.582 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:27.575+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:27.695 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:27.689+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:27.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:27.694+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:27.863 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:27.855+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:28.017 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:28.010+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:28.114 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:28.108+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:28.115 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:28.110+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:28.189 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:28.182+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:28.327 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:28.322+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:28.609 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:28.601+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:28.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:28.667+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:28.678 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:28.673+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:28.833 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:28.824+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:28.984 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:28.977+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:29.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:29.136+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:29.143 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:29.138+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:29.233 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:29.225+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:29.313 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:29.306+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:29.636 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:29.628+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:29.728 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:29.721+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:29.730 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:29.724+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:29.838 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:29.829+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:29.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:29.986+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:30.183 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:30.176+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:30.184 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:30.179+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:30.247 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:30.238+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:30.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:30.286+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:30.612 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:30.603+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:30.764 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:30.759+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:30.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:30.762+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:30.885 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:30.877+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:31.007 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:30.998+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:31.181 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:31.176+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:31.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:31.177+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:31.258 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:31.249+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:31.259 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:31.254+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:31.596 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:31.587+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:31.745 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:31.739+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:31.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:31.744+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:31.857 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:31.848+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:31.989 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:31.981+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:32.225 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:32.217+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:32.229 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:32.224+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:32.231 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:32.226+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:32.255 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:32.248+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:32.560 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:32.552+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:32.708 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:32.701+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:32.709 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:32.704+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:32.867 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:32.859+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:32.979 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:32.969+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:33.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:33.180+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:33.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:33.252+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:33.259 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:33.254+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:33.261 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:33.255+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:33.577 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:33.568+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:33.669 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:33.663+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:33.671 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:33.665+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:33.917 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:33.909+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:33.956 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:33.948+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:34.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:34.189+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:34.267 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:34.262+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:34.298 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:34.292+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:34.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:34.295+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:34.603 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:34.595+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:34.694 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:34.688+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:34.694 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:34.689+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:34.935 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:34.928+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:34.938 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:34.930+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:35.183 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:35.175+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:35.275 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:35.270+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:35.276 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:35.271+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:35.303 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:35.297+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:35.476 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:16:35.457+0000 20e22700 -1 RGWWatcher::handle_error cookie 5435222272 err (107) Transport endpoint is not connected 2019-11-29T18:16:35.573 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:35.564+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:35.678 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:35.669+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:35.679 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:35.673+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:35.902 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:35.893+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:35.977 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:35.968+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:36.153 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:36.143+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:36.266 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:36.261+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:36.323 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:36.317+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:36.324 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:36.319+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:36.582 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:36.573+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:36.636 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:36.630+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:36.637 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:36.632+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:36.923 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:36.915+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:37.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:37.019+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:37.158 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:37.149+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:37.270 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:37.257+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:37.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:37.325+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:37.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:37.327+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:37.632 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:37.623+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:37.641 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:37.635+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:37.646 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:37.641+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:37.939 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:37.930+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:38.023 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:38.014+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:38.184 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:38.176+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:38.222 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:38.216+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:38.381 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:38.373+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:38.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:38.377+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:38.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:38.625+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:38.636 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:38.629+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:38.647 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:38.640+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:38.948 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:38.941+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:39.031 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:39.023+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:39.163 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:39.155+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:39.226 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:39.218+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:39.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:39.342+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:39.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:39.344+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:39.677 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:39.670+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:39.680 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:39.673+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:39.681 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:39.675+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:39.924 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:39.916+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:40.069 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:40.060+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:40.186 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:40.178+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:40.219 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:40.213+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:40.293 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:16:40.286+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 47sec e435 v435) 2019-11-29T18:16:40.335 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:40.328+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:40.335 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:40.330+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:40.679 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:40.672+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:40.682 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:40.676+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:40.684 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:40.676+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:40.923 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:40.915+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:41.074 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:41.067+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:41.182 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:41.174+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:41.209 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:41.203+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:41.322 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:41.316+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:41.322 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:41.317+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:41.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:41.644+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:41.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:41.647+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:41.703 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:41.695+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:41.884 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:41.875+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:42.096 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:42.088+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:42.195 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:42.188+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:42.224 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:42.216+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:42.349 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:42.342+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:42.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:42.344+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:42.632 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:42.625+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:42.634 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:42.627+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:42.702 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:42.695+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:42.898 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:42.889+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:43.146 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:43.138+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:43.185 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:43.166+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:43.200 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:43.193+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:43.334 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:43.327+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:43.336 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:43.330+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:43.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:43.601+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:43.609 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:43.604+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:43.717 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:43.708+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:43.948 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:43.941+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:44.133 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:44.125+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:44.160 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:44.154+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:44.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:16:44.227+0000 1694c700 -1 osd.4 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:16.696906+0000 front 2019-11-29T18:15:16.760296+0000 (oldest deadline 2019-11-29T18:16:02.541480+0000) 2019-11-29T18:16:44.362 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:44.353+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:44.363 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:44.356+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:44.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:44.645+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:44.656 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:44.649+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:44.763 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:44.756+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:44.988 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:44.980+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:45.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:45.132+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:45.149 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:45.142+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:45.308 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:16:45.300+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 47sec e435 v435) 2019-11-29T18:16:45.359 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:45.353+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:45.359 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:45.354+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:45.682 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:45.674+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:45.682 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:45.677+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:45.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:45.800+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:45.957 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:45.950+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:46.109 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:46.101+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:46.159 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:46.152+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:46.406 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:46.400+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:46.408 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:46.403+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:46.671 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:46.662+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:46.673 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:46.666+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:46.765 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:46.758+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:46.934 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:46.928+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:47.128 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:47.121+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:47.153 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:47.146+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:47.377 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:47.370+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:47.380 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:47.375+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:47.688 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:47.681+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:47.692 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:47.686+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:47.730 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:47.722+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:47.922 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:47.915+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:48.117 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:48.109+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:48.177 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:48.169+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:48.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:48.374+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:48.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:48.377+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:48.710 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:48.699+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:48.713 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:48.705+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:48.781 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:48.768+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:48.964 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:48.957+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:49.137 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:49.129+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:49.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:49.132+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:49.371 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:49.361+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:49.373 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:49.364+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:49.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:16:49.741+0000 1694c700 -1 osd.6 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.749076+0000 front 2019-11-29T18:15:21.747065+0000 (oldest deadline 2019-11-29T18:16:04.462670+0000) 2019-11-29T18:16:49.760 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:49.753+0000 1694c700 -1 osd.0 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:15:36.869862+0000 (oldest deadline 2019-11-29T18:16:16.869862+0000) 2019-11-29T18:16:49.761 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:49.755+0000 1694c700 -1 osd.0 435 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e434) 2019-11-29T18:16:49.929 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:16:49.921+0000 1694c700 -1 osd.7 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.511612+0000 front 2019-11-29T18:15:18.512543+0000 (oldest deadline 2019-11-29T18:16:01.981915+0000) 2019-11-29T18:16:50.100 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:16:50.092+0000 1694c700 -1 osd.5 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:21.334873+0000 front 2019-11-29T18:15:21.172559+0000 (oldest deadline 2019-11-29T18:16:05.790491+0000) 2019-11-29T18:16:50.177 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:16:50.171+0000 1694c700 -1 osd.2 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.617940+0000 front 2019-11-29T18:15:18.843649+0000 (oldest deadline 2019-11-29T18:16:02.654065+0000) 2019-11-29T18:16:50.345 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:16:50.338+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 47sec e435 v435) 2019-11-29T18:16:50.412 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:50.405+0000 1694c700 -1 osd.3 435 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:15:18.861006+0000 front 2019-11-29T18:15:18.763100+0000 (oldest deadline 2019-11-29T18:16:04.568351+0000) 2019-11-29T18:16:50.413 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:50.407+0000 1694c700 -1 osd.3 435 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:50.986 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:16:50.978+0000 1694c700 -1 osd.0 436 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:109457 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e434) 2019-11-29T18:16:51.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:51.368+0000 1694c700 -1 osd.3 436 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:81070 7.2 7:46cb1ec1:reshard::reshard.0000000003:head [call lock.lock in=70b] snapc 0=[] ondisk+write+known_if_redirected e432) 2019-11-29T18:16:52.434 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:16:52.414+0000 1694c700 -1 osd.3 437 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:109502 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5435222272] snapc 0=[] ondisk+write+known_if_redirected e435) 2019-11-29T18:17:44.149 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:17:44.133+0000 1694c700 -1 osd.0 441 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=92 ondisk+retry+read+known_if_redirected e439) 2019-11-29T18:17:45.691 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:17:45.346+0000 1694c700 -1 osd.0 441 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=92 ondisk+retry+read+known_if_redirected e439) 2019-11-29T18:18:36.327 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:36.320+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:37.329 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:37.323+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:37.825 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:37.818+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:38.374 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:38.368+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:38.439 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:38.432+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:38.581 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:38.572+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:38.861 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:38.854+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:39.425 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:39.418+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:39.454 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:39.447+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:39.628 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:39.621+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:39.911 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:39.905+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:39.933 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:39.924+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:39.946 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:39.938+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:40.446 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:40.440+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:40.479 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:40.471+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:40.597 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:40.591+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:40.869 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:40.862+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:40.893 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:40.884+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:40.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:40.931+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:40.971 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:40.963+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:41.450 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:41.441+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:41.479 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:41.473+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:41.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:41.580+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:41.857 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:41.850+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:41.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:41.868+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:41.966 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:41.958+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:41.999 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:41.991+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:42.431 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:42.423+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:42.477 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:42.470+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:42.565 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:42.558+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:42.846 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:42.840+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:42.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:42.868+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:42.994 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:42.986+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:43.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:43.017+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:43.447 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:43.439+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:43.468 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:43.458+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:43.566 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:43.561+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:43.810 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:43.805+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:43.857 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:43.849+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:43.964 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:43.956+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:44.013 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:44.005+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:44.498 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:44.493+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:44.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:44.493+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:44.616 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:44.609+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:44.853 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:44.847+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:44.863 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:44.856+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:44.969 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:44.961+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:45.067 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:45.059+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:45.466 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:45.461+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:45.502 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:45.495+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:45.592 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:45.586+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:45.814 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:45.808+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:45.821 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:45.813+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:45.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:45.932+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:46.097 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:46.090+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:46.479 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:46.472+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:46.515 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:46.509+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:46.552 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:46.546+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:46.791 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:46.786+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:46.867 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:46.860+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:46.914 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:46.906+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:47.136 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:47.128+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:47.474 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:47.467+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:47.512 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:47.506+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:47.532 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:47.524+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:47.781 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:47.775+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:47.893 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:47.886+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:47.897 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:47.890+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:48.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:48.087+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:48.481 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:48.475+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:48.495 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:48.489+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:48.546 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:48.539+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:48.812 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:48.806+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:48.864 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:48.857+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:48.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:48.878+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:49.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:49.110+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:49.451 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:49.445+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:49.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:49.495+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:49.517 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:49.511+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:49.807 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:49.802+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:49.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:49.834+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:49.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:49.857+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:50.097 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:50.089+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:50.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:50.424+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:50.527 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:50.519+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:50.535 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:50.528+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:50.814 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:50.806+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:50.841 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:50.836+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:50.854 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:50.846+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:51.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:51.139+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:51.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:51.422+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:51.511 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:51.503+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:51.516 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:51.511+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:51.821 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:51.800+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:51.856 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:51.849+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:51.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:51.878+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:52.160 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:52.152+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:52.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:52.380+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:52.533 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:52.526+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:52.550 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:52.544+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:52.790 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:52.780+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:52.877 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:52.870+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:52.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:52.914+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:53.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:53.134+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:53.357 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:53.351+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:53.550 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:53.544+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:53.601 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:53.594+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:53.762 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:53.755+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:53.863 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:53.858+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:53.927 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:53.920+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:54.122 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:54.114+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:54.325 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:54.319+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:54.551 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:54.543+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:54.651 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:54.642+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:54.778 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:54.770+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:54.879 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:54.872+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:54.934 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:54.919+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:55.127 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:55.120+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:55.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:55.372+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:55.575 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:55.567+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:55.721 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:55.696+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:55.828 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:55.820+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:55.840 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:55.833+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:55.895 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:55.888+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:56.131 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:56.124+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:56.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:56.377+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:56.619 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:56.611+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:56.749 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:56.740+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:56.868 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:56.861+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:56.873 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:56.866+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:56.915 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:56.907+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:57.185 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:57.177+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:57.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:18:57.384+0000 1694c700 -1 osd.2 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.134609+0000 front 2019-11-29T18:17:56.247389+0000 (oldest deadline 2019-11-29T18:18:38.459352+0000) 2019-11-29T18:18:57.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:57.597+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:57.708 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:57.700+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:57.826 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:57.820+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:57.895 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:57.885+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:57.906 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:57.898+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:58.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:58.181+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:58.565 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:58.558+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:58.748 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:58.740+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:58.822 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:58.815+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:58.872 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:58.861+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:18:58.883 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:58.876+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:59.213 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:18:59.205+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:18:59.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:18:59.595+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:18:59.737 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:18:59.729+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:18:59.844 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:18:59.837+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:18:59.845 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:18:59.838+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:18:59.867 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:18:59.860+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:00.253 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:00.246+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:00.615 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:00.607+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:00.771 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:00.764+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:00.809 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:00.803+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:00.890 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:00.882+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:00.901 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:00.893+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:01.254 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:01.246+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:01.664 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:01.656+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:01.757 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:01.748+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:01.802 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:01.793+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:01.905 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:01.897+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:01.908 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:01.899+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:02.242 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:02.231+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:02.672 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:02.665+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:02.794 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:02.787+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:02.807 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:02.799+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:02.926 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:02.918+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:02.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:02.943+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:03.245 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:03.235+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:03.685 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:03.679+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:03.760 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:03.752+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:03.793 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:03.784+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:03.917 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:03.908+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:03.920 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:03.911+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:04.245 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:04.236+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:04.699 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:04.691+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:04.723 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:04.716+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:04.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:04.770+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:04.880 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:04.872+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:04.881 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:04.872+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:05.225 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:05.215+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:05.692 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:05.686+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:05.761 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:05.752+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:05.772 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:05.766+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:05.896 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:05.888+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:05.937 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:05.927+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:06.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:06.252+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:06.664 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:06.657+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:06.740 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:06.732+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:06.788 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:06.782+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:06.867 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:06.858+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:06.946 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:06.936+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:07.251 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:07.240+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:07.674 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:07.666+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:07.772 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:07.765+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:07.791 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:07.782+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:07.854 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:07.845+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:07.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:07.899+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:08.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:08.254+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:08.686 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:08.677+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:08.818 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:08.811+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:08.821 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:08.812+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:08.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:08.877+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:08.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:08.902+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:09.247 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:09.234+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:09.722 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:09.715+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:09.826 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:09.819+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:09.852 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:09.843+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:09.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:19:09.876+0000 1694c700 -1 osd.5 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.455235+0000 front 2019-11-29T18:17:56.385461+0000 (oldest deadline 2019-11-29T18:18:39.826327+0000) 2019-11-29T18:19:09.899 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:19:09.889+0000 1694c700 -1 osd.4 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:55.477937+0000 front 2019-11-29T18:17:56.531821+0000 (oldest deadline 2019-11-29T18:18:40.143020+0000) 2019-11-29T18:19:10.245 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:19:10.237+0000 1694c700 -1 osd.7 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.141606+0000 front 2019-11-29T18:17:54.648140+0000 (oldest deadline 2019-11-29T18:18:39.916924+0000) 2019-11-29T18:19:10.762 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:19:10.756+0000 1694c700 -1 osd.1 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.356975+0000 front 2019-11-29T18:17:55.597570+0000 (oldest deadline 2019-11-29T18:18:35.557733+0000) 2019-11-29T18:19:10.775 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:19:10.768+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e441 v441) 2019-11-29T18:19:10.838 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:19:10.832+0000 1694c700 -1 osd.3 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:56.140957+0000 front 2019-11-29T18:17:54.173070+0000 (oldest deadline 2019-11-29T18:18:37.618459+0000) 2019-11-29T18:19:10.868 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:10.860+0000 1694c700 -1 osd.6 441 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:17:52.538372+0000 front 2019-11-29T18:17:52.324157+0000 (oldest deadline 2019-11-29T18:18:37.491353+0000) 2019-11-29T18:19:11.393 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:06:24.281 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.396 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:06:24.285 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.416 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:06:24.189 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.419 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:06:24.192 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.444 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:06:24.332 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.451 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:06:24.336 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.682 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:06:24.455 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.741 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:06:24.629 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:11.758 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:06:24.646 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:19:47.425 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:19:47.413+0000 114ab700 -1 osd.6 446 build_incremental_map_msg missing incremental map 445 2019-11-29T18:20:08.139 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:08.130+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:09.113 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:09.102+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:09.373 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:09.364+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:09.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:09.379+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:10.065 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:10.058+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:10.205 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:10.197+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:10.345 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:10.340+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:10.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:10.410+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:11.038 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:11.033+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:11.048 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:11.020+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:11.213 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:11.206+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:11.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:11.326+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:11.385 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:11.376+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:11.541 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:11.533+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:12.000 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:20:11.947+0000 20e22700 -1 RGWWatcher::handle_error cookie 900276080 err (107) Transport endpoint is not connected 2019-11-29T18:20:12.016 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:12.008+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:12.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:12.070+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:12.220 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:12.212+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:12.363 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:12.355+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:12.389 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:12.381+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:12.576 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:12.569+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:13.046 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:13.039+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:13.108 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:13.103+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:13.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:13.181+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:13.326 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:13.318+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:13.369 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:13.354+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:13.585 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:13.577+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:14.012 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:14.004+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:14.072 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:14.064+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:14.224 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:14.216+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:14.284 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:14.275+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:14.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:14.380+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:14.584 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:14.576+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:15.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:15.035+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:15.051 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:15.043+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:15.276 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:15.268+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:15.322 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:15.314+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:15.395 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:15.388+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:15.550 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:15.536+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:16.034 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:16.024+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:16.045 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:16.041+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:16.048 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:16.042+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:16.279 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:16.272+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:16.359 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:16.351+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:16.430 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:16.424+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:16.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:16.494+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:17.036 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:17.029+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:17.053 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:17.048+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:17.057 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:17.049+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:17.270 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:17.258+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:17.385 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:17.374+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:17.461 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:17.455+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:17.507 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:17.497+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:18.015 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:18.009+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:18.015 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:18.007+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:18.066 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:18.061+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:18.288 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:18.280+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:18.358 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:18.350+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:18.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:18.501+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:18.558 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:18.550+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:19.005 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:18.996+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:19.041 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:19.032+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:19.059 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:19.053+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:19.256 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:19.248+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:19.363 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:19.345+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:19.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:19.486+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:19.562 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:19.553+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:19.979 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:19.971+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:20.063 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:20.054+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:20.095 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:20.088+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:20.224 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:20.215+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:20.312 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:20.303+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:20.535 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:20.527+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:20.542 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:20.535+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:20.963 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:20.954+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:21.051 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:21.047+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:21.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:21.059+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:21.220 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:21.211+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:21.335 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:21.327+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:21.546 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:21.537+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:21.579 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:21.573+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:21.978 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:21.970+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:22.055 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:22.048+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:22.098 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:22.093+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:22.259 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:22.250+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:22.355 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:22.346+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:22.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:22.507+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:22.567 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:22.560+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:22.937 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:22.929+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:23.068 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:23.061+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:23.080 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:23.074+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:23.264 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:23.255+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:23.321 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:23.313+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:23.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:23.499+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:23.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:23.554+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:23.916 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:23.906+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:24.035 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:24.029+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:24.037 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:24.031+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:24.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:24.069+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:24.304 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:24.295+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:24.377 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:24.356+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:24.506 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:24.490+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:24.559 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:24.554+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:24.915 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:24.907+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:25.046 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:25.037+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:25.051 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:25.045+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:25.101 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:25.093+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:25.327 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:25.319+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:25.359 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:25.350+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:25.479 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:25.469+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:25.605 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:25.599+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:25.913 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:25.904+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:26.070 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:26.061+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:26.071 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:26.065+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:26.092 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:26.086+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:26.309 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:26.297+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:26.322 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:39.210 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.333 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:39.221 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.345 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:26.337+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:26.371 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:39.258 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.405 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:39.293 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.423 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:39.311 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.442 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:39.214 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.443 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:26.434+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:26.444 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:39.217 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.541 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:39.429 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.601 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:39.372 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:26.613 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:26.601+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:26.966 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:26.957+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:27.108 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:27.102+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:27.112 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:27.106+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:27.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:27.126+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:27.289 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:27.282+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:27.371 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:27.363+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:27.459 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:27.446+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:27.651 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:27.643+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:28.017 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:28.008+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:28.140 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:28.132+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:28.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:28.139+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:28.149 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:28.143+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:28.326 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:28.318+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:28.415 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:28.405+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:28.477 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:28.470+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:28.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:28.613+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:29.036 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:29.027+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:29.174 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:29.167+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:29.199 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:29.193+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:29.211 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:29.203+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:29.338 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:29.328+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:29.403 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:29.395+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:29.453 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:29.445+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:29.672 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:29.663+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:30.044 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:30.021+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:30.224 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:30.217+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:30.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:30.255+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:30.274 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:30.260+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:30.312 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:30.304+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:30.380 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:30.371+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:30.436 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:30.426+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:30.708 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:30.690+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:31.051 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:31.043+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:31.242 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:31.236+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:31.266 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:31.258+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:31.267 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:31.261+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:31.346 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:31.339+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:31.399 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:31.385+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:31.421 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:31.414+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:31.672 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:31.661+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:32.038 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:32.030+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:32.232 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:32.225+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:32.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:32.228+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:32.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:32.231+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:32.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:32.300+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:32.390 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:32.382+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:32.422 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:32.415+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:32.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:32.672+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:33.060 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:33.051+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:33.234 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:33.227+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:33.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:33.262+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:33.272 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:33.266+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:33.347 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:33.337+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:33.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:33.409+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:33.421 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:33.412+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:33.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:33.660+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:34.060 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:34.052+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:34.224 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:34.217+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:34.272 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:34.264+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:34.275 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:34.270+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:34.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:34.300+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:34.412 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:34.405+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:34.469 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:34.461+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:34.667 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:34.659+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:35.052 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:35.045+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:35.263 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:35.257+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:35.265 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:35.258+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:35.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:35.280+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:35.291 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:35.285+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:35.454 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:35.447+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:35.466 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:35.459+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:35.687 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:35.680+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:36.087 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:36.075+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:36.230 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:36.222+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:36.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:36.261+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:36.270 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:36.264+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:36.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:36.300+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:36.435 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:36.428+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:36.500 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:36.486+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:36.683 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:36.672+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:37.111 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:37.103+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:37.273 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:37.265+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:37.283 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:37.277+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:37.287 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:37.281+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:37.315 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:37.308+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:37.410 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:37.402+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:37.516 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:37.508+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:37.706 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:37.698+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:38.160 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:38.151+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:38.272 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:38.264+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:38.273 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:38.266+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:38.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:38.273+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:38.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:38.305+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:38.402 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:38.394+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:38.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:38.510+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:38.671 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:38.663+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:39.115 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:39.107+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:39.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:39.253+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:39.260 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:39.254+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:39.288 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:39.281+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:39.317 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:39.311+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:39.360 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:39.352+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:39.524 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:39.516+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:39.686 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:39.677+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:40.084 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:40.076+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:40.251 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:40.244+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:40.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:40.248+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:40.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:40.254+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:40.274 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:40.266+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:40.325 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:40.316+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:40.490 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:40.483+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:40.670 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:40.659+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:41.118 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:41.109+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:41.211 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:41.203+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:41.228 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:41.219+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:41.229 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:41.222+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:41.307 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:41.301+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:41.338 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:41.330+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:41.379 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:20:41.369+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e446 v446) 2019-11-29T18:20:41.392 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:54.280 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.429 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:54.317 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.437 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:54.325 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.445 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:54.333 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.464 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:07:54.352 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.473 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:54.241 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.477 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:07:54.361 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.477 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:54.246 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.533 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:41.526+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:41.588 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:07:54.360 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:20:41.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:41.677+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:41.730 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:20:41.678+0000 20e22700 -1 RGWWatcher::handle_error cookie 5485812976 err (107) Transport endpoint is not connected 2019-11-29T18:20:42.090 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:42.083+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:42.176 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:42.169+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:42.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:42.224+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:42.232 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:42.226+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:42.362 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:42.342+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:42.363 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:42.350+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:42.569 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:42.561+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:42.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:42.659+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:43.078 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:43.071+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:43.156 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:43.147+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:43.282 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:43.273+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:43.283 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:43.275+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:43.359 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:43.351+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:43.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:43.392+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:43.546 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:43.534+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:43.691 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:43.686+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:44.091 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:44.083+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:44.123 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:44.115+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:44.238 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:44.234+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:44.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:44.236+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:44.347 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:44.339+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:44.400 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:44.393+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:44.520 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:44.512+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:44.719 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:44.712+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:45.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:45.109+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:45.124 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:45.116+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:45.217 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:45.210+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:45.219 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:45.212+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:45.321 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:45.313+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:45.356 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:45.350+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:45.556 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:45.548+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:45.732 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:45.726+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:46.084 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:46.074+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:46.094 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:46.086+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:46.178 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:46.171+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:46.179 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:46.174+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:46.356 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:46.344+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:46.387 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:20:46.379+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e446 v446) 2019-11-29T18:20:46.393 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:46.384+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:46.553 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:46.545+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:46.707 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:46.697+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:46.707 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:46.699+0000 1694c700 -1 osd.3 446 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:47.045 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:47.037+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:47.114 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:47.106+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:47.146 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:47.138+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:47.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:47.142+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:47.352 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:47.343+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:47.367 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:47.362+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:47.590 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:47.582+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:47.731 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:47.725+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:47.737 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:47.729+0000 1694c700 -1 osd.3 446 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:48.085 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:48.078+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:48.086 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:48.079+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:48.179 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:48.168+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:48.180 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:48.174+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:48.331 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:48.324+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:48.406 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:48.401+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:48.624 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:48.616+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:48.772 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:48.766+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:48.773 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:48.768+0000 1694c700 -1 osd.3 446 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:49.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:49.109+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:49.123 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:49.114+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:49.170 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:49.164+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:49.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:49.166+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:49.379 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:49.371+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:49.452 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:49.446+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:49.643 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:49.635+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:49.729 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:49.723+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:49.731 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:49.726+0000 1694c700 -1 osd.3 446 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:50.163 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:50.155+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:50.171 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:50.163+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:50.210 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:50.204+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:50.212 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:50.206+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:50.349 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:50.341+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:50.410 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:50.405+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:50.656 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:50.647+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:50.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:50.731+0000 1694c700 -1 osd.3 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.790464+0000 front 2019-11-29T18:19:25.788416+0000 (oldest deadline 2019-11-29T18:20:09.138317+0000) 2019-11-29T18:20:50.740 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:50.733+0000 1694c700 -1 osd.3 446 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:51.131 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:20:51.124+0000 1694c700 -1 osd.7 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.213819+0000 front 2019-11-29T18:19:27.274714+0000 (oldest deadline 2019-11-29T18:20:09.979465+0000) 2019-11-29T18:20:51.159 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:20:51.152+0000 1694c700 -1 osd.4 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.419433+0000 front 2019-11-29T18:19:24.209294+0000 (oldest deadline 2019-11-29T18:20:07.968683+0000) 2019-11-29T18:20:51.252 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:51.244+0000 1694c700 -1 osd.0 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:19:43.871739+0000 (oldest deadline 2019-11-29T18:20:23.871739+0000) 2019-11-29T18:20:51.254 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:20:51.247+0000 1694c700 -1 osd.0 446 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e445) 2019-11-29T18:20:51.363 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:20:51.353+0000 1694c700 -1 osd.5 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:24.727556+0000 front 2019-11-29T18:19:24.734123+0000 (oldest deadline 2019-11-29T18:20:08.765347+0000) 2019-11-29T18:20:51.374 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:20:51.369+0000 1694c700 -1 osd.2 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:25.687679+0000 front 2019-11-29T18:19:25.866217+0000 (oldest deadline 2019-11-29T18:20:10.961796+0000) 2019-11-29T18:20:51.416 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:20:51.412+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e446 v446) 2019-11-29T18:20:51.642 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:20:51.636+0000 1694c700 -1 osd.6 446 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:19:27.277800+0000 front 2019-11-29T18:19:26.955042+0000 (oldest deadline 2019-11-29T18:20:10.940536+0000) 2019-11-29T18:20:52.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:51.970+0000 1694c700 -1 osd.3 447 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:20:53.121 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:20:53.017+0000 1694c700 -1 osd.3 448 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:109886 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5485812976] snapc 0=[] ondisk+write+known_if_redirected e446) 2019-11-29T18:21:50.517 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:21:50.498+0000 1694c700 -1 osd.0 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109643 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected e449) 2019-11-29T18:21:51.661 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:21:51.575+0000 1694c700 -1 osd.0 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:109643 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected e449) 2019-11-29T18:22:43.571 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:43.564+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:43.992 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:43.984+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:44.109 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:44.101+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:44.618 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:44.601+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:44.619 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:44.603+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:44.851 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:44.844+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:44.866 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:44.859+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:44.998 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:44.990+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:45.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:45.155+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:45.625 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:45.617+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:45.639 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:45.632+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:45.694 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:45.688+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:45.839 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:45.831+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:45.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:45.840+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:45.954 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:45.946+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:46.175 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:46.168+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:46.601 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:46.594+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:46.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:46.624+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:46.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:46.717+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:46.809 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:46.802+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:46.898 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:46.891+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:46.949 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:46.941+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:47.200 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:47.191+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:47.562 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:47.554+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:47.649 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:47.642+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:47.732 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:47.725+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:47.860 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:47.853+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:47.925 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:47.916+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:47.928 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:47.920+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:48.224 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:48.216+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:48.596 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:48.590+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:48.673 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:48.666+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:48.744 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:48.737+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:48.854 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:48.846+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:48.893 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:48.885+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:48.920 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:48.910+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:49.198 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:49.191+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:49.605 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:49.598+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:49.651 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:49.644+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:49.716 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:49.707+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:49.837 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:49.828+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:49.875 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:49.864+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:49.911 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:49.904+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:50.211 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:50.203+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:50.605 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:50.597+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:50.682 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:50.676+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:50.703 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:50.695+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:50.793 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:50.783+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:50.929 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:50.919+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:50.934 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:50.925+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:51.202 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:51.193+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:51.580 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:51.571+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:51.642 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:51.635+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:51.682 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:51.675+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:51.804 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:51.794+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:51.940 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:51.931+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:51.985 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:51.976+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:52.205 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:52.196+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:52.587 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:52.580+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:52.675 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:52.667+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:52.720 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:52.713+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:52.758 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:52.749+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:52.952 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:52.945+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:52.994 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:52.985+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:53.243 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:53.234+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:53.620 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:53.613+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:53.705 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:53.699+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:53.753 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:53.746+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:53.798 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:53.790+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:53.914 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:53.905+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:53.983 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:53.975+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:54.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:54.253+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:54.597 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:54.589+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:54.685 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:54.678+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:54.770 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:54.762+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:54.791 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:54.785+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:54.899 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:54.892+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:55.018 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:55.009+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:55.241 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:55.232+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:55.567 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:55.561+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:55.682 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:55.675+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:55.793 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:55.785+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:55.844 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:55.837+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:55.856 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:55.847+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:56.025 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:56.017+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:56.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:56.252+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:56.590 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:56.582+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:56.685 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:56.676+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:56.766 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:56.758+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:56.818 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:56.809+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:56.828 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:56.822+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:56.983 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:56.974+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:57.304 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:57.295+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:57.622 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:57.614+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:57.721 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:57.713+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:57.814 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:57.806+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:57.832 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:57.823+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:57.860 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:57.854+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:57.965 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:57.956+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:58.354 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:58.345+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:58.639 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:58.631+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:58.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:58.673+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:58.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:58.839+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:58.864 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:58.856+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:58.878 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:58.870+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:22:59.015 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:22:59.006+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:22:59.356 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:22:59.346+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:22:59.654 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:22:59.649+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:22:59.666 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:22:59.661+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:22:59.831 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:22:59.824+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:22:59.847 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:22:59.840+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:22:59.857 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:22:59.852+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:00.043 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:00.036+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:00.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:00.389+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:00.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:00.620+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:00.675 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:00.669+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:00.813 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:00.805+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:00.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:00.812+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:00.882 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:00.876+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:01.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:01.053+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:01.406 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:01.399+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:01.655 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:01.648+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:01.702 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:01.696+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:01.830 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:01.822+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:01.872 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:01.864+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:01.885 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:01.876+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:02.105 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:02.097+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:02.367 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:02.360+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:02.679 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:02.673+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:02.753 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:02.747+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:02.879 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:02.872+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:02.894 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:02.886+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:02.915 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:02.906+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:03.131 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:03.123+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:03.335 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:03.327+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:03.653 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:03.647+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:03.800 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:03.794+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:03.852 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:03.845+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:03.869 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:03.862+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:03.915 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:03.907+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:04.089 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:04.081+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:04.387 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:04.379+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:04.617 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:04.611+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:04.833 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:04.825+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:04.858 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:04.851+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:04.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:04.861+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:04.878 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:04.870+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:05.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:05.063+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:05.389 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:05.380+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:05.572 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:05.566+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:05.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:05.841+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:05.860 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:05.853+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:05.883 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:05.876+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:05.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:05.891+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:06.101 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:06.094+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:06.422 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:06.415+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:06.598 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:06.588+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:06.804 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:06.799+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:06.868 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:06.861+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:06.905 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:06.897+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:06.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:06.911+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:07.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:07.100+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:07.419 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:07.412+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:07.607 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:07.601+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:07.828 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:07.821+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:07.884 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:07.879+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:07.935 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:07.927+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:07.960 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:07.952+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:08.142 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:08.134+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:08.442 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:08.434+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:08.642 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:08.635+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:08.864 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:08.857+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:08.868 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:08.863+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:08.917 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:08.907+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:08.955 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:08.947+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:09.179 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:09.171+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:09.423 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:09.415+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:09.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:09.617+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:09.819 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:09.812+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:09.841 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:09.835+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:09.897 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:09.889+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:09.953 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:09.946+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:10.208 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:10.201+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:10.427 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:10.420+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:10.670 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:10.664+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:10.838 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:10.830+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:10.890 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:10.885+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:10.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:10.913+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:10.933 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:10.924+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:11.189 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:11.181+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:11.422 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:11.414+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:11.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:11.630+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:11.840 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:11.834+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:11.853 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:11.847+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:11.888 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:11.880+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:11.920 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:11.913+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:12.161 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:12.154+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:12.405 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:12.396+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:12.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:12.598+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:12.875 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:12.869+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:12.888 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:12.883+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:12.893 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:12.886+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:12.922 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:12.915+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:13.164 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:13.156+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:13.380 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:13.372+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:13.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:13.599+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:13.845 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:13.839+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:13.861 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:13.854+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:13.896 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:13.888+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:13.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:13.913+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:13.996 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:10:26.884 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.003 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:10:26.892 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.012 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:10:26.900 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.017 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:10:26.904 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.023 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:10:26.796 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.038 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:10:26.811 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.123 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:14.117+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:14.165 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:10:27.053 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.213 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:10:27.101 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.251 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:10:27.024 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:23:14.420 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:14.413+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:14.591 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:14.585+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:14.868 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:14.861+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:14.876 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:14.871+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:14.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:14.875+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:14.962 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:14.953+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:15.119 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:15.111+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:15.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:15.386+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:15.623 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:15.619+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:15.838 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:15.830+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:15.917 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:15.911+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:15.917 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:15.912+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:15.981 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:23:15.974+0000 1694c700 -1 osd.4 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:00.745842+0000 front 2019-11-29T18:22:00.852743+0000 (oldest deadline 2019-11-29T18:22:44.839719+0000) 2019-11-29T18:23:16.130 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:23:16.122+0000 1694c700 -1 osd.6 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:01.636368+0000 front 2019-11-29T18:22:01.637674+0000 (oldest deadline 2019-11-29T18:22:43.757487+0000) 2019-11-29T18:23:16.348 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:23:16.340+0000 1694c700 -1 osd.7 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:21:59.761192+0000 front 2019-11-29T18:21:59.730606+0000 (oldest deadline 2019-11-29T18:22:43.235083+0000) 2019-11-29T18:23:16.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:16.614+0000 1694c700 -1 osd.3 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:03.058816+0000 front 2019-11-29T18:21:58.863504+0000 (oldest deadline 2019-11-29T18:22:42.857474+0000) 2019-11-29T18:23:16.844 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:23:16.835+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e451 v451) 2019-11-29T18:23:16.869 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:23:16.858+0000 1694c700 -1 osd.5 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.543572+0000 front 2019-11-29T18:22:02.545532+0000 (oldest deadline 2019-11-29T18:22:43.974689+0000) 2019-11-29T18:23:16.882 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:23:16.878+0000 1694c700 -1 osd.2 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.025933+0000 front 2019-11-29T18:22:01.976295+0000 (oldest deadline 2019-11-29T18:22:45.435399+0000) 2019-11-29T18:23:16.899 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:23:16.894+0000 1694c700 -1 osd.1 451 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:22:02.041268+0000 front 2019-11-29T18:22:02.475640+0000 (oldest deadline 2019-11-29T18:22:44.265770+0000) 2019-11-29T18:23:51.104 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:51.095+0000 34271700 -1 osd.3 455 build_incremental_map_msg missing incremental map 455 2019-11-29T18:23:51.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:51.355+0000 35a74700 -1 osd.3 455 build_incremental_map_msg missing incremental map 455 2019-11-29T18:23:51.462 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:51.447+0000 37a78700 -1 osd.3 455 build_incremental_map_msg missing incremental map 455 2019-11-29T18:23:53.209 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:23:53.204+0000 124ad700 -1 osd.3 456 build_incremental_map_msg missing incremental map 455 2019-11-29T18:24:14.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:14.500+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:15.341 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:15.330+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:15.475 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:15.468+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:15.535 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:15.527+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:16.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:16.376+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:16.461 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:16.453+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:16.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:16.483+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:16.494 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:16.486+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:16.535 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:16.521+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:17.405 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:17.398+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:17.476 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:17.468+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:17.477 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:17.470+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:17.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:17.475+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:17.514 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:17.506+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:17.886 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:17.878+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:18.390 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:18.383+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:18.441 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:18.432+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:18.503 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:18.496+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:18.525 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:18.516+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:18.547 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:18.537+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:18.878 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:18.869+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:19.373 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:19.367+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:19.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:19.421+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:19.489 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:19.481+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:19.509 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:19.501+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:19.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:19.559+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:19.899 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:19.892+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:20.331 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:20.324+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:20.446 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:20.439+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:20.482 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:20.474+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:20.509 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:24:20.358+0000 20e22700 -1 RGWWatcher::handle_error cookie 5430117936 err (107) Transport endpoint is not connected 2019-11-29T18:24:20.516 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:20.509+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:20.602 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:20.594+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:20.934 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:20.926+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:21.376 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:21.369+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:21.426 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:21.417+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:21.515 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:21.507+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:21.523 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:21.515+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:21.566 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:21.559+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:21.970 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:21.957+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:22.338 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:22.332+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:22.400 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:22.392+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:22.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:22.489+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:22.549 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:22.540+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:22.549 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:22.541+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:22.685 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:22.678+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:22.997 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:22.990+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:23.355 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:23.337+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:23.398 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:23.390+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:23.452 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:23.444+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:23.558 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:23.550+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:23.560 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:23.552+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:23.661 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:23.655+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:24.048 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:24.041+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:24.372 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:24.364+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:24.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:24.380+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:24.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:24.499+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:24.535 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:24.528+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:24.549 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:24.542+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:24.677 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:24.671+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:25.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:25.018+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:25.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:25.342+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:25.401 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:25.394+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:25.503 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:25.495+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:25.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:25.500+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:25.578 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:25.571+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:25.672 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:25.661+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:26.060 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:26.052+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:26.398 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:26.391+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:26.401 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:26.393+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:26.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:26.474+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:26.532 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:26.513+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:26.560 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:26.553+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:26.709 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:26.703+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:27.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:27.086+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:27.369 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:27.361+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:27.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:27.379+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:27.448 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:27.440+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:27.532 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:27.525+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:27.553 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:27.545+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:27.674 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:27.669+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:28.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:28.043+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:28.396 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:28.390+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:28.408 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:28.401+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:28.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:28.447+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:28.547 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:28.538+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:28.548 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:28.540+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:28.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:28.636+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:28.644 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:28.638+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:29.022 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:29.014+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:29.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:29.373+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:29.384 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:29.377+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:29.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:29.410+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:29.506 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:29.498+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:29.509 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:29.502+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:29.660 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:29.651+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:29.661 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:29.652+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:30.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:30.037+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:30.389 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:30.382+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:30.395 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:30.387+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:30.413 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:30.407+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:30.481 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:30.473+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:30.510 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:30.502+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:30.689 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:30.683+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:30.690 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:30.685+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:31.016 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:31.008+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:31.357 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:31.352+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:31.404 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:31.398+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:31.407 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:31.394+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:31.480 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:31.472+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:31.505 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:31.497+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:31.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:31.648+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:31.655 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:31.650+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:32.000 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:31.993+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:32.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:32.374+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:32.404 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:32.398+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:32.432 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:32.424+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:32.487 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:32.476+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:32.492 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:32.482+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:32.696 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:32.688+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:32.697 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:32.691+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:32.973 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:32.963+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:33.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:33.348+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:33.438 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:33.430+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:33.444 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:33.439+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:33.475 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:33.467+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:33.516 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:33.509+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:33.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:33.658+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:33.667 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:33.662+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:33.966 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:33.954+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:34.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:34.343+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:34.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:34.407+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:34.460 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:34.452+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:34.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:34.473+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:34.484 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:34.475+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:34.642 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:34.635+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:34.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:34.638+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:35.014 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:35.006+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:35.386 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:35.376+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:35.386 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:35.381+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:35.461 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:35.453+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:35.488 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:35.479+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:35.515 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:35.508+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:35.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:35.643+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:35.654 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:35.644+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:36.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:36.030+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:36.369 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:36.362+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:36.417 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:36.406+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:36.480 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:36.473+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:36.494 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:36.486+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:36.497 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:36.491+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:36.664 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:36.658+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:36.666 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:36.662+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:37.023 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:37.014+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:37.332 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:37.324+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:37.461 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:37.456+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:37.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:37.492+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:37.537 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:37.515+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:37.537 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:37.526+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:37.629 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:37.616+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:37.629 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:37.620+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:38.038 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:38.030+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:38.338 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:38.330+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:38.429 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:38.420+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:38.506 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:38.501+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:38.511 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:38.505+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:38.563 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:38.555+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:38.659 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:38.651+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:38.663 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:38.655+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:39.091 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:39.084+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:39.360 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:39.352+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:39.387 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:39.381+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:39.534 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:39.526+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:39.535 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:39.526+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:39.556 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:39.550+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:39.645 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:39.641+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:39.648 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:39.643+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:40.143 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:40.135+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:40.375 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:40.362+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:40.404 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:40.397+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:40.514 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:40.506+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:40.516 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:40.510+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:40.552 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:40.545+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:40.649 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:40.643+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:40.650 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:40.645+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:41.110 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:41.102+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:41.349 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:41.337+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:41.450 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:41.443+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:41.481 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:41.474+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:41.533 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:41.526+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:41.561 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:41.555+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:41.652 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:41.647+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:41.653 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:41.649+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:42.155 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:42.148+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:42.330 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:42.325+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:42.470 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:42.462+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:42.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:42.479+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:42.580 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:42.572+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:42.591 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:42.585+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:42.706 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:42.700+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:42.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:42.702+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:43.135 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:43.127+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:43.368 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:43.361+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:43.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:43.479+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:43.493 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:43.485+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:43.562 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:43.553+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:43.599 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:43.581+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:43.707 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:43.701+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:43.708 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:43.703+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:44.125 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:44.116+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:44.340 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:44.333+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:44.454 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:44.445+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:44.478 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:44.469+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:44.530 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:44.520+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:44.612 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:44.598+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:44.731 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:44.723+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:44.732 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:44.727+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:45.154 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:45.145+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:45.328 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:45.322+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:45.457 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:45.448+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:45.468 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:45.456+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:45.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:45.512+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:45.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:45.621+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:45.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:45.745+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:45.753 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:45.748+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:46.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:46.181+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:46.305 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:46.298+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:46.446 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:46.437+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:46.460 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:46.442+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:46.539 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:46.530+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:46.630 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:46.624+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:46.747 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:46.741+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:46.749 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:46.742+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:47.094 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:24:47.085+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e456 v456) 2019-11-29T18:24:47.165 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:47.156+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:47.289 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:47.281+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:47.417 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:47.408+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:47.465 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:47.454+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:47.557 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:47.548+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:47.593 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:47.582+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:47.729 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:47.724+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:47.732 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:47.727+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:47.997 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:24:47.428+0000 20e22700 -1 RGWWatcher::handle_error cookie 387251552 err (107) Transport endpoint is not connected 2019-11-29T18:24:48.177 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:48.168+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:48.256 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:48.250+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:48.441 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:48.423+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:48.442 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:48.425+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:48.526 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:48.519+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:48.623 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:48.618+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:48.724 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:48.717+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:48.725 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:48.719+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:49.215 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:49.206+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:49.279 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:49.273+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:49.412 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:49.404+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:49.468 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:49.460+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:49.526 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:49.518+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:49.613 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:49.607+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:49.766 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:49.761+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:49.768 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:49.763+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:50.209 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:50.196+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:50.330 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:50.325+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:50.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:50.441+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:50.452 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:50.444+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:50.527 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:50.518+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:50.631 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:50.626+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:50.792 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:50.785+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:50.793 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:50.788+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:51.227 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:51.214+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:51.383 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:51.376+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:51.487 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:51.478+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:51.491 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:51.482+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:51.501 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:51.493+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:51.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:51.629+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:51.805 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:51.800+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:51.807 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:51.801+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:52.108 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:24:52.102+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e456 v456) 2019-11-29T18:24:52.197 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:52.188+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:52.363 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:52.357+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:52.496 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:52.487+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:52.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:52.527+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:52.555 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:52.546+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:52.614 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:52.609+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:52.617 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:52.612+0000 1694c700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:24:52.778 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:52.767+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:52.778 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:52.771+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:53.218 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:53.209+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:53.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:53.349+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:53.499 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:53.492+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:53.514 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:53.507+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:53.578 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:53.570+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:53.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:53.598+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:53.606 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:53.601+0000 1694c700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:24:53.825 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:53.817+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:53.827 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:53.821+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:54.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:54.172+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:54.334 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:54.326+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:54.495 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:54.489+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:54.528 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:54.522+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:54.545 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:54.539+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:54.607 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:54.602+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:54.608 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:54.604+0000 1694c700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:24:54.830 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:54.822+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:54.831 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:54.826+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:55.183 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:55.175+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:55.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:55.342+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:55.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:55.479+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:55.536 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:55.529+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:55.547 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:55.540+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:55.638 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:55.630+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:55.641 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:55.632+0000 1694c700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:24:55.823 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:55.815+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:55.825 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:55.819+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:56.163 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:56.157+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:56.359 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:56.353+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:56.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:24:56.477+0000 1694c700 -1 osd.6 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:29.206990+0000 front 2019-11-29T18:23:29.160980+0000 (oldest deadline 2019-11-29T18:24:14.438158+0000) 2019-11-29T18:24:56.530 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:24:56.523+0000 1694c700 -1 osd.4 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.093648+0000 front 2019-11-29T18:23:32.028705+0000 (oldest deadline 2019-11-29T18:24:16.092271+0000) 2019-11-29T18:24:56.582 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:24:56.566+0000 1694c700 -1 osd.5 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:33.633452+0000 front 2019-11-29T18:23:33.622467+0000 (oldest deadline 2019-11-29T18:24:15.816291+0000) 2019-11-29T18:24:56.627 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:56.621+0000 1694c700 -1 osd.3 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:30.436234+0000 front 2019-11-29T18:23:30.446709+0000 (oldest deadline 2019-11-29T18:24:14.449489+0000) 2019-11-29T18:24:56.628 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:56.623+0000 1694c700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:24:56.827 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:56.818+0000 1694c700 -1 osd.0 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:23:48.623948+0000 (oldest deadline 2019-11-29T18:24:28.623948+0000) 2019-11-29T18:24:56.829 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:24:56.824+0000 1694c700 -1 osd.0 456 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e455) 2019-11-29T18:24:57.168 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:24:57.161+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e456 v456) 2019-11-29T18:24:57.194 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:24:57.187+0000 1694c700 -1 osd.7 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:32.292935+0000 front 2019-11-29T18:23:32.313496+0000 (oldest deadline 2019-11-29T18:24:17.461875+0000) 2019-11-29T18:24:57.353 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:24:57.347+0000 1694c700 -1 osd.2 456 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:23:28.613989+0000 front 2019-11-29T18:23:28.612711+0000 (oldest deadline 2019-11-29T18:24:14.469439+0000) 2019-11-29T18:24:57.874 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:24:57.767+0000 1694c700 -1 osd.3 457 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110256 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552] snapc 0=[] ondisk+write+known_if_redirected e456) 2019-11-29T18:25:20.506 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:25:20.326+0000 1694c700 -1 osd.1 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110202 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e454) 2019-11-29T18:26:47.156 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:47.147+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:47.370 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:47.364+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:47.473 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:47.464+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:48.178 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:48.171+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:48.334 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:48.329+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:48.413 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:48.408+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:48.457 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:48.449+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:49.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:49.183+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:49.290 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:49.283+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:49.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:49.380+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:49.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:49.502+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:49.945 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:49.930+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:50.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:50.002+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:50.247 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:50.239+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:50.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:50.276+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:50.350 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:50.340+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:50.482 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:50.474+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:50.635 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:50.626+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:50.983 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:50.976+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:51.050 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:51.041+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:51.250 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:51.243+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:51.251 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:51.242+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:51.325 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:51.319+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:51.490 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:51.482+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:51.621 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:51.613+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:52.009 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:52.001+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:52.024 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:52.016+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:52.211 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:52.203+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:52.259 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:52.252+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:52.347 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:52.340+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:52.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:52.487+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:52.590 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:52.582+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:53.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:52.996+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:53.043 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:53.035+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:53.204 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:53.196+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:53.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:53.267+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:53.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:53.294+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:53.517 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:53.510+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:53.571 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:53.563+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:54.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:54.004+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:54.062 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:54.054+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:54.227 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:54.219+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:54.311 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:54.305+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:54.314 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:54.309+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:54.545 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:54.537+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:54.563 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:54.543+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:55.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:55.033+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:55.059 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:55.053+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:55.260 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:55.252+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:55.289 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:55.283+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:55.295 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:55.289+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:55.523 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:55.514+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:55.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:55.536+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:56.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:56.010+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:56.103 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:56.097+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:56.265 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:56.258+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:56.292 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:56.281+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:56.334 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:56.328+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:56.480 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:56.472+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:56.576 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:56.568+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:56.984 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:56.977+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:57.113 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:57.107+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:57.260 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:57.253+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:57.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:57.287+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:57.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:57.381+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:57.509 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:57.501+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:57.593 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:57.586+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:57.950 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:57.941+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:58.093 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:58.086+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:58.230 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:58.221+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:58.264 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:58.256+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:58.409 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:58.403+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:58.470 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:58.462+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:58.637 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:58.630+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:58.981 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:58.959+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:26:59.138 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:26:59.130+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:26:59.237 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:26:59.229+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:26:59.263 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:26:59.257+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:26:59.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:26:59.368+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:26:59.471 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:26:59.463+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:26:59.660 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:26:59.653+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:26:59.929 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:26:59.918+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:00.138 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:00.132+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:00.264 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:00.256+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:00.285 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:00.278+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:00.358 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:00.350+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:00.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:00.421+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:00.700 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:00.692+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:00.940 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:00.932+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:01.173 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:01.167+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:01.268 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:01.262+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:01.307 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:01.300+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:01.320 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:01.314+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:01.458 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:01.450+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:01.738 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:01.730+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:01.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:01.923+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:02.219 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:02.209+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:02.286 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:02.277+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:02.310 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:02.302+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:02.314 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:02.307+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:02.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:02.409+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:02.754 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:02.745+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:02.941 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:02.933+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:03.238 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:03.232+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:03.243 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:03.237+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:03.345 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:03.338+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:03.346 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:03.338+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:03.417 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:03.410+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:03.789 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:03.783+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:03.925 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:03.918+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:04.217 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:04.208+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:04.261 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:04.256+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:04.334 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:04.326+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:04.384 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:04.374+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:04.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:04.422+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:04.750 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:04.744+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:04.947 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:04.938+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:05.184 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:05.177+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:05.315 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:05.307+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:05.342 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:05.336+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:05.362 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:05.354+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:05.437 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:05.428+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:05.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:05.748+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:05.988 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:05.979+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:06.161 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:06.155+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:06.323 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:06.317+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:06.353 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:06.344+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:06.373 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:06.366+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:06.450 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:06.441+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:06.743 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:06.735+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:07.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:07.031+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:07.167 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:07.160+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:07.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:07.339+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:07.357 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:07.348+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:07.422 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:07.412+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:07.423 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:07.417+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:07.768 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:07.757+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:08.044 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:08.036+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:08.128 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:08.121+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:08.325 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:08.319+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:08.376 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:08.366+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:08.437 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:08.429+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:08.449 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:08.442+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:08.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:08.755+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:09.078 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:09.069+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:09.117 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:09.106+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:09.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:09.297+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:09.348 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:09.339+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:09.406 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:09.400+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:09.475 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:09.467+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:09.731 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:09.722+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:10.052 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:10.045+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:10.119 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:10.112+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:10.278 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:10.272+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:10.360 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:10.353+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:10.416 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:10.408+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:10.511 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:10.504+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:10.785 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:10.769+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:11.071 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:11.062+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:11.130 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:11.106+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:11.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:11.225+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:11.395 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:11.389+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:11.401 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:11.393+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:11.520 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:11.511+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:11.783 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:11.775+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:12.057 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:12.048+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:12.155 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:12.149+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:12.262 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:12.255+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:12.354 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:12.347+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:12.413 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:12.406+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:12.542 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:12.535+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:12.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:12.750+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:13.055 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:13.039+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:13.147 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:13.132+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:13.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:13.246+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:13.372 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:13.365+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:13.382 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:13.373+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:13.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:13.547+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:13.713 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:13.706+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:13.741 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:14:26.629 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.754 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:14:26.640 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.757 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:14:26.645 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.758 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:14:26.646 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.768 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:14:26.541 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.782 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:14:26.554 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.975 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:14:26.862 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:13.985 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:14:26.873 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:14.010 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:14.002+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:14.101 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:14.090+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:14.101 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:14:26.874 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:27:14.232 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:14.225+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:14.348 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:14.341+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:14.421 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:14.413+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:14.589 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:14.581+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:14.679 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:14.668+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:15.023 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:15.016+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:15.076 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:15.069+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:15.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:15.198+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:15.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:27:15.325+0000 1694c700 -1 osd.2 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:03.127286+0000 front 2019-11-29T18:26:03.124812+0000 (oldest deadline 2019-11-29T18:26:48.233633+0000) 2019-11-29T18:27:15.463 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:15.453+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:15.552 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:15.543+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:15.675 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:15.668+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:16.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:15.996+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:16.124 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:16.116+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:16.226 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:16.219+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:16.422 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:16.413+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:16.509 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:16.501+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:16.753 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:16.743+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:17.024 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:17.009+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:17.177 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:17.170+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:17.204 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:17.197+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:17.426 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:17.418+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:17.540 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:17.531+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:17.755 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:17.746+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:18.045 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:18.036+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:18.150 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:18.142+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:18.233 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:18.226+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:18.457 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:18.449+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:18.556 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:18.546+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:18.799 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:18.790+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:19.037 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:19.027+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:19.205 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:19.192+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:19.227 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:19.220+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:19.491 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:19.483+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:19.525 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:19.517+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:19.812 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:19.804+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:20.039 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:20.030+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:20.204 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:20.197+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:20.240 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:20.230+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:20.499 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:20.489+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:20.543 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:20.534+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:20.841 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:20.831+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:21.031 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:21.021+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:21.190 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:21.184+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:21.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:21.243+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:21.464 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:21.455+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:21.562 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:21.553+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:21.827 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:21.813+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:22.061 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:27:22.050+0000 1694c700 -1 osd.5 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:04.821480+0000 front 2019-11-29T18:26:04.306313+0000 (oldest deadline 2019-11-29T18:26:49.058078+0000) 2019-11-29T18:27:22.160 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:27:22.153+0000 1694c700 -1 osd.1 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.313316+0000 front 2019-11-29T18:26:05.095091+0000 (oldest deadline 2019-11-29T18:26:49.558615+0000) 2019-11-29T18:27:22.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:22.293+0000 1694c700 -1 osd.3 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.476270+0000 front 2019-11-29T18:26:02.490285+0000 (oldest deadline 2019-11-29T18:26:47.074228+0000) 2019-11-29T18:27:22.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:27:22.508+0000 1694c700 -1 osd.6 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:06.355313+0000 front 2019-11-29T18:26:06.812876+0000 (oldest deadline 2019-11-29T18:26:47.110504+0000) 2019-11-29T18:27:22.520 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:27:22.512+0000 1694c700 -1 osd.4 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:05.208899+0000 front 2019-11-29T18:26:04.840125+0000 (oldest deadline 2019-11-29T18:26:49.965424+0000) 2019-11-29T18:27:22.650 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:27:22.643+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e461 v461) 2019-11-29T18:27:22.830 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:27:22.821+0000 1694c700 -1 osd.7 461 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:26:02.491684+0000 front 2019-11-29T18:26:02.349758+0000 (oldest deadline 2019-11-29T18:26:47.022133+0000) 2019-11-29T18:27:54.730 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:54.534+0000 35a74700 -1 osd.3 465 build_incremental_map_msg missing incremental map 465 2019-11-29T18:27:54.801 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:54.626+0000 34271700 -1 osd.3 465 build_incremental_map_msg missing incremental map 465 2019-11-29T18:27:57.205 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:27:57.199+0000 124ad700 -1 osd.3 467 build_incremental_map_msg missing incremental map 465 2019-11-29T18:28:21.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:21.080+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:22.074 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:22.069+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:22.186 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:22.170+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:22.794 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:22.784+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:23.034 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:23.028+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:23.225 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:23.217+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:23.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:23.255+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:23.786 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:23.778+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:23.827 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:23.819+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:24.032 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:24.021+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:24.035 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:24.029+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:24.235 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:24.227+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:24.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:24.247+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:24.773 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:24.765+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:24.869 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:24.860+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:24.980 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:24.974+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:25.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:25.023+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:25.270 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:25.263+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:25.298 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:25.291+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:25.780 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:25.771+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:25.849 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:25.842+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:26.006 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:26.000+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:26.070 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:26.065+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:26.167 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:26.162+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:26.236 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:26.227+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:26.284 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:26.277+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:26.843 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:26.823+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:26.844 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:26.828+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:27.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:27.008+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:27.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:27.046+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:27.151 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:27.146+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:27.198 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:27.190+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:27.267 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:27.261+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:27.825 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:27.817+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:27.866 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:27.859+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:28.018 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:28.007+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:28.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:28.022+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:28.170 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:28.162+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:28.204 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:28.199+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:28.312 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:28.304+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:28.831 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:28.823+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:28.878 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:28.870+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:29.007 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:29.002+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:29.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:29.041+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:29.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:29.166+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:29.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:29.181+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:29.271 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:29.263+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:29.833 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:29.825+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:29.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:29.924+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:30.008 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:30.002+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:30.023 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:30.017+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:30.149 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:30.144+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:30.174 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:30.167+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:30.306 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:30.298+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:30.825 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:30.817+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:30.954 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:30.945+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:31.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:31.009+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:31.036 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:31.031+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:31.115 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:31.109+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:31.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:31.113+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:31.221 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:31.213+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:31.350 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:31.343+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:31.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:31.809+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:31.990 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:31.983+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:31.994 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:31.988+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:32.024 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:32.018+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:32.103 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:32.089+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:32.104 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:32.091+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:32.192 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:32.183+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:32.328 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:32.319+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:32.813 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:32.804+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:32.989 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:32.980+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:32.994 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:32.987+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:33.006 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:33.000+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:33.134 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:33.125+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:33.135 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:33.127+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:33.246 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:33.225+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:33.383 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:33.374+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:33.768 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:33.760+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:33.987 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:33.982+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:34.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:34.005+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:34.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:34.014+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:34.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:34.094+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:34.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:34.096+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:34.249 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:34.240+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:34.379 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:34.370+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:34.754 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:34.745+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:34.965 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:34.958+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:35.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:34.992+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:35.048 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:35.043+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:35.056 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:35.051+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:35.057 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:35.052+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:35.280 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:35.272+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:35.338 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:35.329+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:35.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:35.719+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:35.976 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:35.967+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:36.006 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:36.000+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:36.057 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:36.051+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:36.059 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:36.055+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:36.094 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:36.089+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:36.255 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:28:36.250+0000 20e22700 -1 RGWWatcher::handle_error cookie 439940224 err (107) Transport endpoint is not connected 2019-11-29T18:28:36.286 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:36.276+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:36.351 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:36.342+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:36.765 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:36.756+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:36.969 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:36.960+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:36.988 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:36.981+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:37.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:37.032+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:37.042 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:37.037+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:37.060 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:37.054+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:37.263 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:37.255+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:37.364 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:37.355+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:37.792 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:37.783+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:37.971 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:37.963+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:38.005 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:38.000+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:38.008 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:38.003+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:38.018 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:38.011+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:38.026 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:38.021+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:38.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:38.273+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:38.377 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:38.368+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:38.844 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:38.835+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:38.937 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:38.928+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:38.989 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:38.975+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:39.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:39.031+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:39.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:39.034+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:39.076 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:39.071+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:39.328 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:39.319+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:39.369 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:39.359+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:39.866 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:39.858+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:39.895 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:39.886+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:39.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:39.950+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:39.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:39.951+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:40.056 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:40.050+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:40.061 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:40.055+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:40.107 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:40.100+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:40.357 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:40.344+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:40.413 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:40.403+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:40.849 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:40.840+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:40.927 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:40.919+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:40.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:40.941+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:40.948 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:40.943+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:41.049 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:41.042+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:41.050 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:41.044+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:41.064 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:41.058+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:41.371 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:41.363+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:41.380 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:41.371+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:41.830 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:41.822+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:41.939 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:41.926+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:41.984 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:41.977+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:41.985 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:41.979+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:42.024 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:42.017+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:42.025 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:42.019+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:42.112 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:42.106+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:42.395 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:42.387+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:42.396 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:42.388+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:42.881 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:42.874+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:42.940 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:42.933+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:43.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:43.011+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:43.016 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:43.012+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:43.063 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:43.057+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:43.067 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:43.062+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:43.095 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:43.089+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:43.430 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:43.422+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:43.440 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:43.433+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:43.860 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:43.854+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:43.920 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:43.909+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:44.000 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:43.994+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:44.001 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:43.995+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:44.086 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:44.080+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:44.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:44.083+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:44.098 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:44.092+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:44.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:44.435+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:44.481 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:44.473+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:44.891 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:44.883+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:44.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:44.895+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:44.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:44.958+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:44.968 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:44.961+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:45.081 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:45.073+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:45.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:45.076+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:45.143 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:45.136+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:45.447 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:45.439+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:45.498 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:45.490+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:45.906 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:45.900+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:45.945 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:45.936+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:45.947 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:45.940+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:45.949 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:45.942+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:46.125 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:46.111+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:46.125 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:46.117+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:46.183 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:46.175+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:46.468 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:46.460+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:46.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:46.510+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:46.917 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:46.903+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:46.925 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:46.918+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:46.925 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:46.918+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:46.934 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:46.922+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:47.129 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:47.121+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:47.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:47.124+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:47.141 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:47.134+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:47.438 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:47.429+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:47.530 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:47.523+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:47.889 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:47.881+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:47.914 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:47.908+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:47.915 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:47.910+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:47.977 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:47.970+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:48.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:48.123+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:48.132 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:48.125+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:48.143 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:48.134+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:48.477 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:48.469+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:48.501 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:48.494+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:48.852 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:48.844+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:48.933 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:48.925+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:48.934 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:48.927+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:49.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:49.005+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:49.103 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:49.097+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:49.105 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:49.099+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:49.163 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:49.156+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:49.465 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:49.457+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:49.499 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:49.491+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:49.853 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:49.833+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:49.956 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:49.949+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:49.959 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:49.953+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:50.029 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:50.021+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:50.133 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:50.127+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:50.135 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:50.130+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:50.169 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:50.160+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:50.519 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:50.511+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:50.521 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:50.513+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:50.813 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:50.804+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:50.972 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:50.964+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:50.975 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:50.969+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:51.050 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:51.043+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:51.170 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:51.154+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:51.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:51.159+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:51.190 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:51.184+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:51.487 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:51.480+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:51.536 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:51.528+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:51.816 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:51.809+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:52.028 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:52.021+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:52.029 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:52.023+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:52.072 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:52.064+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:52.179 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:52.172+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:52.185 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:52.179+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:52.196 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:52.189+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:52.456 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:52.450+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:52.578 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:52.569+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:52.810 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:52.803+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:52.982 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:52.976+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:52.983 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:52.977+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:53.127 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:53.116+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:53.167 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:53.161+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:53.204 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:53.194+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:53.205 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:53.198+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:53.488 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:53.480+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:53.553 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:53.545+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:53.805 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:53.798+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:54.007 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:54.000+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:54.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:54.002+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:54.095 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:54.087+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:54.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:54.158+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:54.168 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:54.162+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:54.181 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:54.174+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:54.490 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:54.482+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:54.543 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:54.535+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:54.787 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:54.779+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:55.056 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:55.048+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:55.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:55.052+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:55.061 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:55.055+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:55.184 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:55.177+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:55.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:55.181+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:55.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:55.185+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:55.528 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:55.520+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:55.594 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:55.586+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:55.831 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:55.825+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:56.059 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:56.050+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:56.087 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:56.080+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:56.088 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:56.081+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:56.185 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:56.178+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:56.233 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:56.226+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:56.237 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:56.230+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:56.582 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:56.572+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:56.612 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:56.604+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:56.847 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:56.839+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:57.074 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:57.067+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:57.134 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:57.126+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:57.135 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:57.129+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:57.207 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:57.200+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:57.237 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:57.227+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:57.242 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:57.231+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:57.567 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:57.559+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:57.605 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:57.597+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:57.809 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:57.802+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:57.896 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:28:57.886+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e467 v467) 2019-11-29T18:28:58.116 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:58.107+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:58.175 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:58.168+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:58.185 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:58.179+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:58.186 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:58.180+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:58.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:58.261+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:58.270 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:58.264+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:58.565 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:58.558+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:58.588 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:58.580+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:58.807 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:58.800+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:28:59.106 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:28:59.098+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:28:59.173 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:28:59.166+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:28:59.177 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:59.170+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:28:59.177 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:28:59.171+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:28:59.236 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:59.229+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:28:59.238 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:28:59.233+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:28:59.556 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:28:59.547+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:28:59.600 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:28:59.592+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:28:59.764 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:28:59.757+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:29:00.070 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:29:00.062+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:29:00.135 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:00.126+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:29:00.135 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:00.128+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:29:00.223 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:29:00.217+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:29:00.250 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:00.243+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:29:00.254 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:00.248+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:29:00.576 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:29:00.569+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:29:00.644 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:29:00.636+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:29:00.793 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:29:00.784+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:29:01.051 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:29:01.040+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:29:01.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:01.141+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:29:01.150 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:01.143+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:29:01.187 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:29:01.179+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:29:01.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:01.252+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:29:01.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:01.254+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:29:01.541 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:29:01.533+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:29:01.679 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:29:01.671+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:29:01.834 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:29:01.822+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:29:02.087 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:29:02.079+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:29:02.153 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:29:02.147+0000 1694c700 -1 osd.2 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:38.950917+0000 front 2019-11-29T18:27:39.080012+0000 (oldest deadline 2019-11-29T18:28:23.562542+0000) 2019-11-29T18:29:02.187 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:02.181+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:29:02.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:02.183+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:29:02.254 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:02.248+0000 1694c700 -1 osd.0 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:27:50.710712+0000 (oldest deadline 2019-11-29T18:28:30.710712+0000) 2019-11-29T18:29:02.255 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:02.250+0000 1694c700 -1 osd.0 467 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e465) 2019-11-29T18:29:02.576 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:29:02.568+0000 1694c700 -1 osd.7 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.592940+0000 front 2019-11-29T18:27:36.674384+0000 (oldest deadline 2019-11-29T18:28:22.466266+0000) 2019-11-29T18:29:02.683 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:29:02.677+0000 1694c700 -1 osd.6 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:36.052130+0000 front 2019-11-29T18:27:36.047404+0000 (oldest deadline 2019-11-29T18:28:21.529626+0000) 2019-11-29T18:29:02.885 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:29:02.868+0000 1694c700 -1 osd.4 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:37.021650+0000 front 2019-11-29T18:27:36.996310+0000 (oldest deadline 2019-11-29T18:28:22.270317+0000) 2019-11-29T18:29:03.036 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:29:03.011+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 45sec e467 v467) 2019-11-29T18:29:03.065 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:29:03.054+0000 1694c700 -1 osd.5 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.338989+0000 front 2019-11-29T18:27:39.825924+0000 (oldest deadline 2019-11-29T18:28:23.721273+0000) 2019-11-29T18:29:03.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:03.172+0000 1694c700 -1 osd.3 467 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:27:39.722552+0000 front 2019-11-29T18:27:39.815221+0000 (oldest deadline 2019-11-29T18:28:20.121768+0000) 2019-11-29T18:29:03.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:03.174+0000 1694c700 -1 osd.3 467 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:29:03.186 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:16:16.074 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.196 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:16:16.083 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.235 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:16:16.007 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.237 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:16:16.010 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.249 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:16:16.134 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.249 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:16:16.136 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.344 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:16:16.232 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.369 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:16:16.138 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.377 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:16:16.260 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:29:03.817 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:29:03.628+0000 1694c700 -1 osd.0 468 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:110593 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e465) 2019-11-29T18:29:04.150 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:29:04.145+0000 1694c700 -1 osd.3 468 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:110615 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 439940224] snapc 0=[] ondisk+write+known_if_redirected e467) 2019-11-29T18:29:34.376 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.369+0000 38279700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.585 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.499+0000 39a7c700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.587 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.502+0000 3927b700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.677 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.544+0000 34271700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.752 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.596+0000 3a27d700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.782 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.629+0000 3b27f700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:34.845 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.644+0000 36275700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:29:35.025 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:29:34.914+0000 3927b700 -1 osd.1 472 build_incremental_map_msg missing incremental map 472 2019-11-29T18:30:54.626 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:54.618+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:54.686 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:54.679+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:54.916 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:54.909+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:30:55.619 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:55.611+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:55.723 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:55.714+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:55.873 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:55.868+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:30:56.262 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:30:56.253+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:30:56.436 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:30:56.430+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:30:56.650 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:56.642+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:56.721 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:56.712+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:56.869 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:56.863+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:30:57.101 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:30:57.095+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:30:57.301 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:30:57.293+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:30:57.445 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:30:57.439+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:30:57.628 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:57.619+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:57.768 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:57.760+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:57.904 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:57.899+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:30:58.126 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:30:58.118+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:30:58.129 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:30:58.123+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:30:58.299 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:30:58.291+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:30:58.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:30:58.395+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:30:58.584 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:58.576+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:58.811 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:58.803+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:58.957 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:58.949+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:30:59.159 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:30:59.153+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:30:59.164 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:30:59.156+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:30:59.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:30:59.265+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:30:59.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:30:59.375+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:30:59.598 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:30:59.590+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:30:59.816 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:30:59.809+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:30:59.936 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:30:59.928+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:00.194 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:00.185+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:00.200 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:00.193+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:00.268 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:00.259+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:00.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:00.383+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:00.633 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:00.627+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:00.818 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:00.811+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:00.968 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:00.960+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:01.214 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:01.206+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:01.246 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:01.237+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:01.315 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:01.306+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:01.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:01.376+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:01.633 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:01.624+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:01.836 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:01.828+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:02.008 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:02.000+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:02.237 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:02.227+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:02.278 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:02.269+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:02.339 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:02.331+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:02.378 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:02.371+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:02.641 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:02.633+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:02.877 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:02.869+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:02.972 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:02.965+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:03.224 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:03.217+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:03.263 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:03.255+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:03.327 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:03.319+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:03.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:03.394+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:03.651 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:03.644+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:03.915 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:03.908+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:03.995 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:03.988+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:04.192 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:04.184+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:04.302 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:04.295+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:04.357 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:04.350+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:04.372 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:04.363+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:04.653 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:04.644+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:04.919 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:04.911+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:05.009 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:05.001+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:05.146 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:05.139+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:05.267 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:05.257+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:05.324 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:05.315+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:05.366 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:05.357+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:05.644 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:05.635+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:05.901 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:05.892+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:06.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:06.033+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:06.140 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:06.133+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:06.271 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:06.261+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:06.326 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:06.318+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:06.357 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:06.350+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:06.645 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:06.635+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:06.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:06.863+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:07.061 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:07.054+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:07.130 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:07.123+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:07.247 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:07.238+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:07.317 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:07.310+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:07.325 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:07.316+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:07.673 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:07.663+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:07.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:07.862+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:08.033 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:08.025+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:08.169 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:08.162+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:08.224 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:08.215+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:08.298 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:08.282+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:08.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:08.300+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:08.676 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:08.667+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:08.920 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:08.910+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:08.991 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:08.984+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:09.144 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:09.137+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:09.196 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:09.186+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:09.251 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:09.242+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:09.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:09.296+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:09.692 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:09.684+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:09.958 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:09.949+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:09.964 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:09.956+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:10.157 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:10.151+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:10.230 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:10.221+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:10.304 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:10.295+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:10.354 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:10.347+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:10.681 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:10.671+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:10.963 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:10.954+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:10.985 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:10.978+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:11.189 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:11.182+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:11.196 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:11.187+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:11.322 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:11.315+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:11.335 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:11.326+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:11.706 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:11.697+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:11.968 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:11.957+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:11.979 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:11.971+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:12.191 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:12.183+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:12.210 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:12.203+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:12.286 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:12.278+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:12.309 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:12.299+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:12.723 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:12.714+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:12.976 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:12.969+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:13.015 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:13.005+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:13.215 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:13.206+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:13.251 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:13.243+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:13.258 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:13.251+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:13.304 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:13.295+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:13.765 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:13.756+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:13.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:13.929+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:14.039 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:14.030+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:14.260 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:14.252+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:14.286 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:14.279+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:14.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:14.280+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:14.302 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:14.294+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:14.793 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:14.785+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:14.960 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:14.950+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:15.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:15.062+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:15.231 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:15.222+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:15.252 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:15.245+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:15.276 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:15.268+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:15.314 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:15.306+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:15.778 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:15.769+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:15.986 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:15.978+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:16.053 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:16.045+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:16.198 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:16.190+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:16.241 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:16.233+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:16.273 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:16.266+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:16.286 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:16.278+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:16.755 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:16.746+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:17.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:17.012+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:17.061 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:17.053+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:17.240 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:17.233+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:17.255 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:17.248+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:17.280 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:17.272+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:17.282 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:17.274+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:17.802 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:17.794+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:18.040 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:18.032+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:18.063 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:18.055+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:18.230 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:18.223+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:18.261 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:18.254+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:18.274 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:18.266+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:18.306 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:18.298+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:18.760 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:18.753+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:19.058 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:19.050+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:19.064 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:19.058+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:19.216 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:19.210+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:19.236 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:19.228+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:19.249 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:19.243+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:19.274 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:19.265+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:19.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:19.809+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:20.059 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:20.051+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:20.080 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:20.073+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:20.201 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:20.196+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:20.202 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:20.193+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:20.228 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:20.220+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:20.267 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:20.259+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:20.867 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:20.859+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:21.068 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:21.061+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:21.107 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:21.101+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:21.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:21.159+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:21.196 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:21.190+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:21.227 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:21.219+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:21.276 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:21.269+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:21.874 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:21.866+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:22.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:22.068+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:22.130 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:22.121+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:22.151 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:22.144+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:22.171 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:22.165+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:22.238 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:22.230+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:22.310 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:22.303+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:22.911 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:22.903+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:23.092 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:23.068+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:23.105 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:23.096+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:23.197 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:23.190+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:23.197 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:23.189+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:23.202 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:23.195+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:23.335 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:23.326+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:23.935 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:23.928+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:24.069 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:24.064+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:24.157 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:24.150+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:24.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:24.183+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:24.190 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:24.185+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:24.191 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:24.183+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:24.296 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:24.288+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:24.805 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:18:37.693 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.809 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:18:37.698 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.817 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:18:37.705 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.823 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:18:37.711 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.834 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:18:37.607 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.854 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:18:37.627 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:24.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:24.897+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:24.984 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:18:37.871 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:25.001 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:18:37.890 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:25.057 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:18:37.829 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:31:25.094 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:25.085+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:25.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:25.188+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:25.207 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:25.199+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:25.232 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:25.226+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:25.242 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:25.234+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:25.257 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:25.249+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:25.906 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:25.898+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:26.072 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:26.066+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:26.183 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:26.177+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:26.194 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:26.188+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:26.222 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:26.216+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:26.252 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:26.244+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:26.297 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:26.288+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:26.882 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:26.874+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:27.058 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:27.052+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:27.193 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:27.188+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:27.199 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:27.191+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:27.244 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:27.234+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:27.256 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:27.251+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:27.321 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:27.313+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:27.872 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:27.864+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:28.030 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:28.023+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:28.153 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:28.148+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:28.207 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:28.200+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:28.232 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:28.222+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:28.247 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:28.241+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:28.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:28.322+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:28.363 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:31:28.357+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e472 v472) 2019-11-29T18:31:28.921 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:28.912+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:29.076 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:29.063+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:29.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:29.142+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:29.185 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:29.178+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:29.216 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:29.209+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:29.303 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:29.296+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:29.328 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:29.320+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:29.902 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:29.893+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:30.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:30.069+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:30.148 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:30.140+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:30.180 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:30.175+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:30.217 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:30.206+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:30.343 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:30.337+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:30.390 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:30.374+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:30.941 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:30.933+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:31.104 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:31.095+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:31.110 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:31.104+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:31.149 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:31.142+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:31.258 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:31.250+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:31.344 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:31.336+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:31.358 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:31.351+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:31.966 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:31.957+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:32.077 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:32.069+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:32.129 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:32.120+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:32.191 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:32.185+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:32.255 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:32.248+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:32.334 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:32.326+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:32.397 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:32.391+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:33.018 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:31:33.010+0000 1694c700 -1 osd.4 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.583053+0000 front 2019-11-29T18:30:14.128833+0000 (oldest deadline 2019-11-29T18:30:54.528594+0000) 2019-11-29T18:31:33.049 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:31:33.043+0000 1694c700 -1 osd.2 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:08.966172+0000 front 2019-11-29T18:30:14.808461+0000 (oldest deadline 2019-11-29T18:30:54.772672+0000) 2019-11-29T18:31:33.122 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:31:33.115+0000 1694c700 -1 osd.7 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.557105+0000 front 2019-11-29T18:30:12.516705+0000 (oldest deadline 2019-11-29T18:30:54.469178+0000) 2019-11-29T18:31:33.223 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:31:33.215+0000 1694c700 -1 osd.5 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:11.591850+0000 front 2019-11-29T18:30:11.702071+0000 (oldest deadline 2019-11-29T18:30:57.388316+0000) 2019-11-29T18:31:33.241 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:31:33.234+0000 1694c700 -1 osd.3 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:14.164504+0000 front 2019-11-29T18:30:14.178824+0000 (oldest deadline 2019-11-29T18:30:55.813094+0000) 2019-11-29T18:31:33.328 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:31:33.321+0000 1694c700 -1 osd.6 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:10.440363+0000 front 2019-11-29T18:30:10.441878+0000 (oldest deadline 2019-11-29T18:30:55.538895+0000) 2019-11-29T18:31:33.372 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:31:33.365+0000 1694c700 -1 osd.1 472 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:30:12.130635+0000 front 2019-11-29T18:30:11.681976+0000 (oldest deadline 2019-11-29T18:30:56.333422+0000) 2019-11-29T18:31:33.445 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:31:33.435+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e472 v472) 2019-11-29T18:32:02.287 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:02.270+0000 37a78700 -1 osd.6 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:02.484 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:02.364+0000 38a7a700 -1 osd.6 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:02.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:02.379+0000 38279700 -1 osd.6 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:02.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:02.481+0000 35a74700 -1 osd.3 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:02.682 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:02.556+0000 38279700 -1 osd.3 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:02.683 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:02.641+0000 114ab700 -1 osd.6 476 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:04.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:04.469+0000 124ad700 -1 osd.3 477 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:04.517 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:04.510+0000 124ad700 -1 osd.6 477 build_incremental_map_msg missing incremental map 476 2019-11-29T18:32:31.054 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:31.045+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:31.615 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:31.598+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:31.708 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:31.702+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:31.770 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:31.762+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:31.892 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:31.885+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:32.068 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:32.060+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:32.645 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:32.637+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:32.673 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:32.658+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:32.737 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:32.727+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:32.880 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:32.871+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:33.039 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:33.031+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:33.622 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:33.616+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:33.648 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:33.641+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:33.699 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:33.691+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:33.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:33.878+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:34.083 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:34.076+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:34.202 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:34.196+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:34.642 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:34.634+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:34.662 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:34.655+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:34.732 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:34.724+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:34.889 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:34.881+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:35.060 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:35.053+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:35.211 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:35.206+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:35.641 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:35.635+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:35.664 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:35.655+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:35.769 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:35.760+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:35.913 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:35.905+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:36.084 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:36.076+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:36.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:36.204+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:36.603 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:36.596+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:36.629 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:36.622+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:36.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:36.717+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:36.899 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:36.879+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:37.050 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:37.043+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:37.249 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:37.238+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:37.607 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:37.599+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:37.671 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:37.663+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:37.733 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:37.724+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:37.877 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:37.868+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:38.073 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:38.066+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:38.209 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:38.203+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:38.627 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:38.620+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:38.699 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:38.689+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:38.715 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:38.707+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:38.898 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:38.891+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:39.051 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:39.043+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:39.241 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:39.235+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:39.635 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:39.628+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:39.695 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:39.687+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:39.707 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:39.693+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:39.865 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:39.858+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:40.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:40.079+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:40.214 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:40.209+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:40.639 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:40.632+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:40.665 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:40.656+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:40.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:40.697+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:40.873 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:40.866+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:41.139 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:41.130+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:41.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:41.232+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:41.241 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:41.235+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:41.605 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:41.597+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:41.630 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:41.622+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:41.723 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:41.715+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:41.849 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:41.841+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:42.173 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:42.164+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:42.264 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:42.256+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:42.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:42.258+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:42.562 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:42.555+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:42.636 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:42.629+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:42.686 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:42.679+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:42.815 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:42.807+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:43.164 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:43.157+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:43.276 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:43.266+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:43.277 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:43.270+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:43.567 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:43.560+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:43.640 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:43.628+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:43.647 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:43.640+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:43.770 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:43.762+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:44.160 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:44.152+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:44.244 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:44.236+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:44.245 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:44.238+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:44.590 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:44.582+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:44.649 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:44.641+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:44.656 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:44.649+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:44.769 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:44.762+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:45.123 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:45.115+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:45.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:45.282+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:45.300 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:45.285+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:45.613 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:45.602+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:45.619 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:45.611+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:45.630 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:45.621+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:45.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:45.759+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:46.123 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:46.115+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:46.287 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:46.279+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:46.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:46.284+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:46.638 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:46.631+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:46.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:46.642+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:46.678 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:46.670+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:46.808 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:46.800+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:47.152 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:47.135+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:47.301 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:47.295+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:47.304 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:47.298+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:47.624 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:47.616+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:47.655 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:47.647+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:47.716 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:47.709+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:47.865 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:47.838+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:48.181 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:48.173+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:48.341 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:48.333+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:48.342 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:48.336+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:48.647 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:48.639+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:48.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:48.642+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:48.754 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:48.746+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:48.835 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:48.828+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:49.188 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:49.181+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:49.387 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:49.379+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:49.388 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:49.382+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:49.636 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:49.629+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:49.689 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:49.682+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:49.740 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:49.733+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:49.828 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:49.821+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:50.161 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:50.153+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:50.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:50.354+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:50.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:50.358+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:50.605 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:50.597+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:50.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:50.643+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:50.784 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:50.775+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:50.792 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:50.785+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:51.125 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:51.113+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:51.369 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:51.363+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:51.373 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:51.368+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:51.589 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:51.582+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:51.657 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:51.651+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:51.760 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:51.752+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:51.844 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:51.837+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:52.098 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:52.089+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:52.376 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:52.369+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:52.377 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:52.371+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:52.556 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:52.548+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:52.636 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:52.630+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:52.807 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:52.800+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:52.861 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:52.853+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:53.075 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:53.067+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:53.408 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:53.401+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:53.409 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:53.404+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:53.530 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:53.522+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:53.589 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:53.584+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:53.837 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:53.829+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:53.912 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:53.904+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:54.087 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:54.081+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:54.374 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:54.366+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:54.376 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:54.369+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:54.542 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:54.534+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:54.641 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:54.635+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:54.829 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:54.817+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:54.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:54.864+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:55.050 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:55.043+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:55.321 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:55.314+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:32:55.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:55.353+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:55.365 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:55.359+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:55.498 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:55.490+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:55.625 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:55.619+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:55.828 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:55.820+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:55.852 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:55.844+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:56.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:56.053+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:56.304 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:56.297+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:32:56.415 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:56.409+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:56.418 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:56.413+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:56.452 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:56.445+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:56.627 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:56.620+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:56.813 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:56.802+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:56.849 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:56.842+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:57.060 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:57.052+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:57.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:57.323+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:32:57.394 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:57.387+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:57.395 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:57.389+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:57.421 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:57.410+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:57.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:57.591+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:57.671 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:32:57.652+0000 20e22700 -1 RGWWatcher::handle_error cookie 5473550048 err (107) Transport endpoint is not connected 2019-11-29T18:32:57.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:57.796+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:57.810 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:57.803+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:58.048 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:58.039+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:58.307 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:58.300+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:32:58.358 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:58.350+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:58.359 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:58.353+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:58.421 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:58.414+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:58.566 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:58.560+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:58.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:58.794+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:32:58.803 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:58.795+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:59.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:32:59.043+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:32:59.326 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:32:59.320+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:32:59.381 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:59.375+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:32:59.384 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:32:59.379+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:32:59.389 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:32:59.381+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:32:59.540 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:32:59.534+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:32:59.762 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:32:59.753+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:32:59.803 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:32:59.793+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:00.053 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:00.044+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:00.321 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:00.308+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:00.401 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:00.387+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:00.422 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:00.416+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:00.424 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:00.418+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:00.568 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:00.562+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:00.795 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:00.779+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:00.822 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:00.806+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:01.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:01.068+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:01.272 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:01.265+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:01.385 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:01.378+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:01.386 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:01.381+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:01.410 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:01.401+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:01.570 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:01.556+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:01.793 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:01.783+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:01.855 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:01.846+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:02.123 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:02.114+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:02.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:02.298+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:02.306 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:02.301+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:02.385 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:02.380+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:02.389 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:02.384+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:02.424 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:02.415+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:02.552 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:02.546+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:02.760 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:02.752+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:02.872 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:02.862+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:03.127 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:03.118+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:03.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:03.338+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:03.346 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:03.339+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:03.412 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:03.394+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:03.412 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:03.398+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:03.429 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:03.421+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:03.529 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:03.519+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:03.738 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:03.729+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:03.792 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:33:03.786+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e477 v477) 2019-11-29T18:33:03.890 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:03.881+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:04.089 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:04.081+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:04.316 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:04.309+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:04.319 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:04.314+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:04.392 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:04.383+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:04.422 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:04.415+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:04.424 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:04.418+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:04.578 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:04.572+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:04.758 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:04.749+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:04.890 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:04.881+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:05.067 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:05.059+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:05.341 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:05.334+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:05.342 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:05.336+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:05.407 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:05.398+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:05.439 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:05.430+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:05.440 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:05.434+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:05.534 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:05.527+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:05.771 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:05.760+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:05.896 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:05.887+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:06.120 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:06.111+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:06.299 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:06.292+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:06.300 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:06.295+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:06.398 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:06.392+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:06.399 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:06.394+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:06.421 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:06.411+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:06.547 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:06.540+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:06.746 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:06.736+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:06.943 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:06.934+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:07.099 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:07.090+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:07.303 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:07.297+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:07.306 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:07.301+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:07.380 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:07.372+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:07.400 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:07.392+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:07.402 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:07.396+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:07.522 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:07.515+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:07.729 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:07.720+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:07.937 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:07.928+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:08.084 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:08.075+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:08.351 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:08.346+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:08.354 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:08.348+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:08.361 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:08.353+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:08.362 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:08.356+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:08.363 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:08.357+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:08.519 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:08.512+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:08.744 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:08.736+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:08.805 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:33:08.801+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e477 v477) 2019-11-29T18:33:08.991 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:08.982+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:09.076 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:09.068+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:09.337 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:09.331+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:09.338 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:09.333+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:09.341 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:09.334+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:09.341 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:09.337+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:09.393 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:09.385+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:09.493 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:09.485+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:09.741 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:09.728+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:10.009 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:09.996+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:10.102 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:10.094+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:10.335 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:10.327+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:10.336 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:10.330+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:10.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:10.350+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:10.357 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:10.353+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:10.427 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:10.419+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:10.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:10.516+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:10.768 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:10.760+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:11.052 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:11.044+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:11.068 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:11.058+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:11.329 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:11.324+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:11.330 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:11.325+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:11.355 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:11.348+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:11.356 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:11.351+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:11.386 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:11.379+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:11.534 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:11.526+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:11.778 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:11.770+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:12.010 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:12.002+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:12.120 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:12.112+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:12.328 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:12.320+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:12.329 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:12.324+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:12.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:12.386+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:12.405 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:12.389+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:12.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:12.417+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:12.526 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:12.520+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:12.757 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:12.734+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:13.035 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:13.027+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:13.125 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:33:13.118+0000 1694c700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:46.778473+0000 front 2019-11-29T18:31:46.750652+0000 (oldest deadline 2019-11-29T18:32:30.814861+0000) 2019-11-29T18:33:13.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:13.365+0000 1694c700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:50.342370+0000 front 2019-11-29T18:31:50.371077+0000 (oldest deadline 2019-11-29T18:32:54.999886+0000) 2019-11-29T18:33:13.376 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:13.367+0000 1694c700 -1 osd.3 477 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:13.434 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:13.426+0000 1694c700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:32:00.226876+0000 (oldest deadline 2019-11-29T18:32:40.226876+0000) 2019-11-29T18:33:13.435 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:33:13.430+0000 1694c700 -1 osd.0 477 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111004 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e476) 2019-11-29T18:33:13.455 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:33:13.447+0000 1694c700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:45.095615+0000 front 2019-11-29T18:31:45.108986+0000 (oldest deadline 2019-11-29T18:32:30.838071+0000) 2019-11-29T18:33:13.498 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:33:13.491+0000 1694c700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.172290+0000 front 2019-11-29T18:31:47.175621+0000 (oldest deadline 2019-11-29T18:32:31.253270+0000) 2019-11-29T18:33:13.784 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:33:13.776+0000 1694c700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:48.941607+0000 front 2019-11-29T18:31:48.920942+0000 (oldest deadline 2019-11-29T18:32:31.795719+0000) 2019-11-29T18:33:13.856 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:33:13.840+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 44sec e477 v477) 2019-11-29T18:33:14.070 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:33:14.063+0000 1694c700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:31:47.992374+0000 front 2019-11-29T18:31:48.042683+0000 (oldest deadline 2019-11-29T18:32:30.884576+0000) 2019-11-29T18:33:14.493 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:14.426+0000 1694c700 -1 osd.3 478 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:33:15.502 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:33:15.452+0000 1694c700 -1 osd.3 479 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111049 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5473550048 gen 1] snapc 0=[] ondisk+write+known_if_redirected e477) 2019-11-29T18:34:09.858 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:34:09.462+0000 1694c700 -1 osd.0 482 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=105 ondisk+retry+read+known_if_redirected e481) 2019-11-29T18:34:10.574 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:34:10.526+0000 1694c700 -1 osd.0 482 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=105 ondisk+retry+read+known_if_redirected e481) 2019-11-29T18:34:11.866 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:34:11.707+0000 1694c700 -1 osd.0 482 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=105 ondisk+retry+read+known_if_redirected e481) 2019-11-29T18:35:03.703 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:03.695+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:03.886 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:03.879+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:04.728 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:04.720+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:04.903 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:04.896+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:05.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:05.744+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:05.873 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:05.863+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:05.874 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:05.868+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:06.173 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:06.164+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:06.345 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:06.339+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:06.769 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:06.760+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:06.871 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:06.864+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:06.883 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:06.878+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:07.213 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:07.205+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:07.353 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:07.347+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:07.571 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:07.563+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:07.761 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:07.753+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:07.863 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:07.854+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:07.910 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:07.905+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:08.222 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:08.214+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:08.374 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:08.368+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:08.624 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:08.616+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:08.719 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:08.712+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:08.879 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:08.871+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:08.920 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:08.915+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:09.188 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:09.180+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:09.285 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:09.279+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:09.380 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:09.373+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:09.592 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:09.583+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:09.686 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:09.676+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:09.895 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:09.888+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:09.927 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:09.920+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:10.222 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:10.203+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:10.310 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:10.303+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:10.417 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:10.411+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:10.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:10.541+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:10.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:10.725+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:10.854 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:10.849+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:10.967 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:10.959+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:11.248 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:11.240+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:11.323 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:11.318+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:11.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:11.380+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:11.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:11.503+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:11.717 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:11.707+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:11.893 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:11.886+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:11.928 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:11.920+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:12.266 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:12.259+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:12.317 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:12.309+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:12.363 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:12.357+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:12.491 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:12.482+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:12.742 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:12.734+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:12.910 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:12.904+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:12.957 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:12.949+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:13.245 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:13.237+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:13.343 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:13.336+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:13.410 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:13.402+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:13.481 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:13.472+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:13.702 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:13.695+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:13.876 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:13.870+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:13.912 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:13.904+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:14.224 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:14.216+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:14.362 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:14.350+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:14.380 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:14.375+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:14.483 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:14.474+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:14.751 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:14.743+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:14.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:14.841+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:14.963 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:14.955+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:15.236 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:15.222+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:15.409 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:15.400+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:15.428 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:15.421+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:15.473 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:15.465+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:15.783 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:15.775+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:15.855 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:15.850+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:15.969 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:15.961+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:16.286 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:16.278+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:16.397 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:16.391+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:16.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:16.436+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:16.475 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:16.470+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:16.837 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:16.828+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:16.894 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:16.888+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:17.002 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:16.994+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:17.337 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:17.330+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:17.416 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:17.409+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:17.421 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:17.415+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:17.439 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:17.433+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:17.884 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:17.878+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:17.885 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:17.877+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:18.036 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:18.028+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:18.394 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:18.381+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:18.443 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:18.436+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:18.462 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:18.456+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:18.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:18.486+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:18.930 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:18.922+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:18.936 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:18.930+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:19.047 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:19.039+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:19.412 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:19.405+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:19.465 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:19.459+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:19.471 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:19.466+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:19.499 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:19.491+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:19.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:19.897+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:19.927 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:19.922+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:20.066 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:20.059+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:20.388 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:20.377+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:20.431 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:20.426+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:20.490 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:20.482+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:20.502 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:20.496+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:20.900 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:20.894+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:20.956 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:20.948+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:21.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:21.086+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:21.382 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:21.374+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:21.472 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:21.466+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:21.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:21.468+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:21.475 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:21.467+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:21.917 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:21.909+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:21.968 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:21.960+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:22.129 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:22.122+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:22.410 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:22.402+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:22.455 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:22.450+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:22.497 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:22.492+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:22.526 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:22.516+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:22.885 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:22.877+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:22.951 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:22.943+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:23.113 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:23.105+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:23.401 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:23.392+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:23.468 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:23.462+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:23.478 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:23.471+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:23.518 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:23.510+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:23.845 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:23.840+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:23.930 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:23.922+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:24.107 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:24.099+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:24.436 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:24.431+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:24.439 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:24.431+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:24.467 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:24.462+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:24.502 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:24.493+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:24.547 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:37.436 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.569 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:37.458 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.604 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:37.492 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.634 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:37.521 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.685 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:37.457 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.689 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:37.462 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.825 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:37.714 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.835 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:24.829+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:24.850 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:37.737 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:24.901 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:24.893+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:24.953 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:37.726 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:25.138 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:25.132+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:25.440 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:25.433+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:25.456 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:25.450+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:25.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:25.468+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:25.509 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:25.502+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:25.820 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:25.813+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:25.918 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:25.911+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:26.133 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:26.126+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:26.476 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:26.471+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:26.487 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:26.481+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:26.495 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:26.487+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:26.562 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:26.555+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:26.806 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:26.799+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:26.913 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:26.904+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:27.183 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:27.173+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:27.448 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:27.440+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:27.483 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:27.477+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:27.505 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:27.496+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:27.586 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:27.579+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:27.830 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:27.824+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:27.925 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:27.917+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:28.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:28.152+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:28.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:28.419+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:28.451 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:28.445+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:28.494 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:28.486+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:28.618 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:28.611+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:28.860 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:28.854+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:28.911 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:28.903+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:29.136 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:29.128+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:29.429 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:29.423+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:29.453 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:29.444+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:29.457 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:29.451+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:29.601 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:29.593+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:29.622 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:42.511 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.629 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:42.517 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.637 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:42.525 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.642 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:42.529 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.648 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:42.421 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.658 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:42.430 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.663 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:22:42.551 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.671 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:22:42.559 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.748 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:22:42.521 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:35:29.829 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:29.824+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:29.911 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:29.904+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:30.091 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:30.083+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:30.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:30.384+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:30.411 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:30.403+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:30.502 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:30.494+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:30.644 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:30.636+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:30.868 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:30.863+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:30.895 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:30.887+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:31.071 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:31.063+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:31.398 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:31.390+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:31.466 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:31.457+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:31.494 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:31.486+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:31.625 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:31.616+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:31.859 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:31.852+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:31.864 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:31.855+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:32.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:32.067+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:32.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:32.349+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:32.456 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:32.449+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:32.511 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:32.504+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:32.635 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:32.628+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:32.818 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:32.811+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:32.820 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:32.812+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:33.085 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:33.077+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:33.413 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:33.400+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:33.482 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:33.475+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:33.492 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:33.483+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:33.627 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:33.618+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:33.848 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:33.841+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:33.848 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:33.834+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:34.070 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:34.061+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:34.422 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:34.415+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:34.456 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:34.448+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:34.492 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:34.484+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:34.662 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:34.654+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:34.856 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:34.849+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:34.876 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:34.867+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:35.089 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:35.080+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:35.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:35.429+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:35.469 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:35.455+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:35.530 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:35.522+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:35.650 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:35.642+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:35.832 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:35.823+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:35.847 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:35.839+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:36.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:36.089+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:36.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:36.416+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:36.424 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:36.415+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:36.513 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:36.505+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:36.688 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:36.679+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:36.795 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:36.787+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:36.809 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:36.803+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:37.060 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:37.051+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:37.421 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:37.414+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:37.447 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:37.430+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:37.475 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:37.469+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:37.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:37.692+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:37.778 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:37.768+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:37.840 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:37.833+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:38.027 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:38.019+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:38.403 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:38.394+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:38.441 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:38.434+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:38.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:38.462+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:38.741 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:35:38.733+0000 1694c700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.161938+0000 front 2019-11-29T18:34:22.502106+0000 (oldest deadline 2019-11-29T18:35:07.281999+0000) 2019-11-29T18:35:38.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:35:38.812+0000 1694c700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.982983+0000 front 2019-11-29T18:34:20.914171+0000 (oldest deadline 2019-11-29T18:35:03.142951+0000) 2019-11-29T18:35:38.872 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:38.865+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:35:39.077 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:35:39.068+0000 1694c700 -1 osd.6 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.945775+0000 front 2019-11-29T18:34:20.306108+0000 (oldest deadline 2019-11-29T18:35:03.476528+0000) 2019-11-29T18:35:39.370 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:35:39.362+0000 1694c700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.896572+0000 front 2019-11-29T18:34:21.909071+0000 (oldest deadline 2019-11-29T18:35:05.357554+0000) 2019-11-29T18:35:39.397 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:35:39.389+0000 1694c700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:22.750054+0000 front 2019-11-29T18:34:22.751866+0000 (oldest deadline 2019-11-29T18:35:08.545164+0000) 2019-11-29T18:35:39.476 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:35:39.469+0000 1694c700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:21.500288+0000 front 2019-11-29T18:34:20.842529+0000 (oldest deadline 2019-11-29T18:35:05.600131+0000) 2019-11-29T18:35:39.578 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:35:39.568+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 45sec e482 v482) 2019-11-29T18:35:39.865 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:35:39.859+0000 1694c700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:34:20.975668+0000 front 2019-11-29T18:34:21.502920+0000 (oldest deadline 2019-11-29T18:35:05.520412+0000) 2019-11-29T18:36:43.038 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:43.033+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:44.047 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:44.041+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:45.052 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:45.046+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:46.068 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:46.063+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:47.081 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:47.075+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:48.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:48.083+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:49.089 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:49.084+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:50.048 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:50.042+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:51.097 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:51.091+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:52.050 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:52.045+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:53.060 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:53.055+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:54.031 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:54.026+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:55.026 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:55.021+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:56.077 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:56.071+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:57.045 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:57.040+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:58.006 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:58.001+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:36:58.996 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:36:58.991+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:00.045 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:00.040+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:01.009 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:01.004+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:01.966 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:01.961+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:02.924 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:02.918+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:03.910 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:03.905+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:04.888 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:04.882+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:05.558 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:37:05.505+0000 20e22700 -1 RGWWatcher::handle_error cookie 387251552 err (107) Transport endpoint is not connected 2019-11-29T18:37:05.860 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:05.853+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:06.839 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:06.833+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:07.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:07.793+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:08.785 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:08.779+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:09.798 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:09.792+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:10.417 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:10.411+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:10.834 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:10.826+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:11.373 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:11.366+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:11.823 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:11.817+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:12.352 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:12.345+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:12.814 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:12.807+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:13.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:13.378+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:13.837 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:13.831+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:14.394 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:14.388+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:14.822 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:14.816+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:15.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:15.417+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:15.807 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:15.801+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:16.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:16.394+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:16.832 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:16.826+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:17.359 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:17.352+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:17.818 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:17.811+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:18.347 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:18.341+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:18.860 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:18.854+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:19.305 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:19.299+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:19.898 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:19.891+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:20.303 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:20.297+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:20.874 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:20.868+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:21.332 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:21.328+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:21.914 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:21.909+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:22.050 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:22.041+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:22.219 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:22.212+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:22.324 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:22.319+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:22.522 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:22.514+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:22.900 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:22.887+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:22.902 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:22.893+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:23.083 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:23.075+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:23.169 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:23.161+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:23.238 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:23.223+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:23.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:23.350+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:23.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:23.351+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:23.481 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:23.474+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:23.919 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:23.912+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:23.920 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:23.915+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:24.086 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:24.079+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:24.208 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:24.186+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:24.241 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:24.235+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:24.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:24.393+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:24.401 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:24.396+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:24.492 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:24.485+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:24.897 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:24.888+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:24.897 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:24.890+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:25.063 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:24:37.951 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.076 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:25.067+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:25.098 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:24:37.985 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.101 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:24:37.989 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.115 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:24:38.003 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.136 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:24:37.909 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.143 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:24:37.915 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.206 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:24:38.093 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.207 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:25.198+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:25.222 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:24:38.111 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.278 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:25.273+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:25.311 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:24:38.084 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:37:25.436 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:25.428+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:25.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:25.430+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:25.482 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:25.472+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:25.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:25.902+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:25.909 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:25.903+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:26.083 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:26.074+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:26.197 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:26.189+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:26.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:26.276+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:26.443 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:26.433+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:26.448 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:26.441+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:26.489 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:26.482+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:26.930 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:26.921+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:26.931 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:26.925+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:27.048 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:27.035+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:27.252 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:27.243+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:27.299 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:27.294+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:27.399 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:27.394+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:27.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:27.397+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:27.513 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:27.503+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:27.971 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:27.967+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:27.973 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:27.969+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:28.076 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:28.068+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:28.254 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:28.245+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:28.337 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:28.330+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:28.426 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:28.420+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:28.427 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:28.422+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:28.551 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:28.542+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:28.975 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:28.968+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:28.977 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:28.971+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:29.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:29.048+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:29.218 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:29.210+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:29.318 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:29.311+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:29.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:29.385+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:29.392 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:29.387+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:29.523 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:29.514+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:29.983 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:29.975+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:29.985 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:29.979+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:30.027 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:30.019+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:30.249 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:30.241+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:30.358 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:30.352+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:30.369 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:30.362+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:30.370 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:30.365+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:30.499 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:30.488+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:30.966 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:30.957+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:30.967 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:30.961+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:31.003 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:30.994+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:31.243 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:31.235+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:31.338 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:31.332+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:31.390 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:31.384+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:31.391 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:31.386+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:31.464 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:31.456+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:31.993 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:31.985+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:31.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:31.992+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:32.003 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:31.999+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:32.207 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:32.198+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:32.355 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:32.349+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:32.416 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:32.410+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:32.416 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:32.412+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:32.435 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:32.427+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:32.992 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:32.984+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:33.046 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:33.038+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:33.048 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:33.042+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:33.250 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:33.239+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:33.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:33.375+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:33.430 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:33.425+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:33.431 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:33.426+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:33.439 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:33.431+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:34.007 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:33.999+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:34.060 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:34.053+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:34.061 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:34.056+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:34.191 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:34.182+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:34.225 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:34.216+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:34.351 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:34.345+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:34.408 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:34.400+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:34.474 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:34.469+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:34.477 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:34.472+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:35.011 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:35.003+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:35.078 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:35.072+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:35.079 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:35.074+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:35.209 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:35.197+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:35.241 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:35.234+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:35.367 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:35.361+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:35.375 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:35.368+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:35.465 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:35.458+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:35.466 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:35.461+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:36.059 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:36.051+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:36.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:36.110+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:36.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:36.113+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:36.174 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:36.167+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:36.288 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:36.277+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:36.361 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:36.352+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:36.402 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:36.395+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:36.472 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:36.465+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:36.473 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:36.467+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:37.101 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:37.094+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:37.172 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:37.163+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:37.173 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:37.168+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:37.215 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:37.207+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:37.336 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:37.324+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:37.388 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:37.381+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:37.424 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:37.417+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:37.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:37.441+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:37.454 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:37.445+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:38.094 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:38.086+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:38.170 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:38.162+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:38.182 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:38.170+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:38.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:38.177+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:38.315 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:38.307+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:38.380 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:38.373+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:38.413 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:38.406+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:38.414 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:38.409+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:38.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:38.418+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:39.102 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:39.094+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:39.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:39.183+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:39.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:39.186+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:39.220 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:39.212+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:39.351 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:39.343+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:39.364 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:39.358+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:39.444 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:39.437+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:39.449 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:39.442+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:39.451 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:39.445+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:40.116 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:40.108+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:40.236 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:40.230+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:40.236 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:40.232+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:40.258 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:40.251+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:40.368 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:40.362+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:40.373 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:40.366+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:40.422 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:40.414+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:40.468 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:40.462+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:40.470 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:40.466+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:41.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:41.110+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:41.201 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:41.195+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:41.207 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:41.201+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:41.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:41.226+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:41.382 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:41.375+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:41.400 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:41.394+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:41.410 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:41.402+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:41.483 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:41.477+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:41.485 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:41.479+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:42.100 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:42.092+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:42.185 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:42.179+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:42.186 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:42.181+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:42.292 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:42.275+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:42.365 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:42.357+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:42.366 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:42.360+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:42.432 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:42.424+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:42.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:42.452+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:42.460 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:42.455+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:43.111 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:43.102+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:43.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:43.164+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:43.176 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:43.170+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:43.243 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:43.235+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:43.351 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:43.345+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:43.410 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:43.402+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:43.481 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:43.476+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:43.482 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:43.474+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:43.483 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:43.479+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:44.101 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:44.094+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:44.169 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:44.163+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:44.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:44.168+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:44.245 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:44.237+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:44.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:44.306+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:44.390 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:44.383+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:44.451 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:44.444+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:44.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:44.520+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:44.527 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:44.522+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:45.075 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:45.058+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:45.133 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:45.126+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:45.134 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:45.129+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:45.235 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:45.227+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:45.312 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:37:45.307+0000 1694c700 -1 osd.2 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.175649+0000 front 2019-11-29T18:36:41.887344+0000 (oldest deadline 2019-11-29T18:37:21.370615+0000) 2019-11-29T18:37:45.370 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:37:45.356+0000 1694c700 -1 osd.6 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:39.905945+0000 front 2019-11-29T18:36:39.965361+0000 (oldest deadline 2019-11-29T18:37:33.202425+0000) 2019-11-29T18:37:45.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:45.492+0000 1694c700 -1 osd.3 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.880389+0000 front 2019-11-29T18:36:38.881477+0000 (oldest deadline 2019-11-29T18:37:22.853667+0000) 2019-11-29T18:37:45.499 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:45.491+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.211058+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:45.499 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:45.494+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:46.031 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:46.024+0000 1694c700 -1 osd.7 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:46.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:46.163+0000 1694c700 -1 osd.0 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:46.172 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:46.167+0000 1694c700 -1 osd.0 487 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:46.295 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:46.279+0000 1694c700 -1 osd.4 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:46.467 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:37:46.456+0000 1694c700 -1 osd.5 487 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:37:46.077644+0000 front 2019-11-29T18:36:38.084399+0000 (oldest deadline 2019-11-29T18:37:22.104053+0000) 2019-11-29T18:37:46.470 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:37:46.464+0000 1694c700 -1 osd.3 487 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111467 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 387251552 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-11-29T18:37:47.159 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:37:47.049+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:37.463387+0000 front 2019-11-29T18:36:41.727676+0000 (oldest deadline 2019-11-29T18:37:21.260543+0000) 2019-11-29T18:37:47.261 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:37:47.255+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:40.403984+0000 front 2019-11-29T18:36:40.308246+0000 (oldest deadline 2019-11-29T18:37:23.101196+0000) 2019-11-29T18:37:47.303 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:47.263+0000 1694c700 -1 osd.0 488 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:36:38.654876+0000 front 2019-11-29T18:36:38.692168+0000 (oldest deadline 2019-11-29T18:37:22.885210+0000) 2019-11-29T18:37:47.304 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:47.265+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7.88d971e9 (undecoded) ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:48.332 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:48.317+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:49.387 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:49.378+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:50.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:50.403+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:51.429 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:51.421+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:52.412 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:52.404+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 19 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:53.388 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:53.381+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 19 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:54.425 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:54.417+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 19 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:55.399 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:55.392+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 19 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:56.458 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:56.450+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:57.476 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:57.470+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:58.512 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:58.505+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:37:59.548 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:37:59.541+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:00.544 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:00.537+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:01.589 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:01.581+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 20 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:02.571 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:02.562+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 21 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:03.605 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:03.598+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 21 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:04.605 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:04.599+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 21 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:05.582 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:05.573+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 21 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:06.554 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:06.546+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 21 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:07.598 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:07.592+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:08.563 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:08.556+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:09.571 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:09.564+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:10.612 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:10.607+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:11.621 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:11.616+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:12.643 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:12.637+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:13.622 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:13.616+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:14.593 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:14.588+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:15.604 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:15.589+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:16.560 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:16.555+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111229 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:18.512 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:18.286+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=110 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:38:19.592 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:38:19.356+0000 1694c700 -1 osd.0 488 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=110 ondisk+retry+read+known_if_redirected e486) 2019-11-29T18:39:12.401 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:12.392+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:12.517 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:12.511+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:12.574 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:12.567+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:13.429 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:13.420+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:13.485 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:13.480+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:13.537 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:13.526+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:13.595 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:13.588+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:13.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:13.957+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:14.466 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:14.458+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:14.471 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:14.464+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:14.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:14.540+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:14.635 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:14.630+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:14.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:14.968+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:15.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:15.427+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:15.480 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:15.474+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:15.584 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:15.576+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:15.658 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:15.653+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:15.702 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:15.697+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:16.012 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:16.005+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:16.246 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:16.238+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:16.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:16.459+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:16.497 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:16.492+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:16.543 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:16.537+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:16.636 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:16.630+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:16.712 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:16.706+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:17.018 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:17.010+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:17.291 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:17.281+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:17.522 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:17.514+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:17.532 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:17.527+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:17.577 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:17.567+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:17.669 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:17.665+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:17.708 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:17.700+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:17.994 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:17.982+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:18.262 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:18.254+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:18.517 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:18.500+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:18.557 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:18.549+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:18.568 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:18.561+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:18.696 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:18.690+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:18.703 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:18.697+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:18.971 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:18.963+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:19.310 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:19.302+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:19.538 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:19.529+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:19.562 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:19.553+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:19.574 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:19.567+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:19.666 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:19.660+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:19.735 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:19.729+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:19.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:19.942+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:20.365 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:20.352+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:20.570 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:20.562+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:20.578 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:20.570+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:20.599 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:20.584+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:20.702 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:20.696+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:20.744 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:20.738+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:20.926 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:20.919+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:21.404 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:21.395+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:21.543 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:21.534+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:21.602 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:21.594+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:21.606 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:21.599+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:21.733 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:21.726+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:21.756 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:21.750+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:21.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:21.894+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:22.367 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:22.359+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:22.531 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:22.523+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:22.589 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:22.580+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:22.620 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:22.612+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:22.755 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:22.749+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:22.784 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:22.776+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:22.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:22.879+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:23.407 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:23.399+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:23.561 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:23.553+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:23.581 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:23.570+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:23.665 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:23.657+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:23.797 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:23.790+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:23.822 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:23.817+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:23.889 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:23.879+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:24.459 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:24.449+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:24.552 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:24.543+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:24.600 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:24.587+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:24.710 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:24.702+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:24.774 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:24.767+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:24.781 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:24.773+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:24.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:24.922+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:25.466 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:25.457+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:25.551 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:25.537+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:25.602 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:25.594+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:25.737 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:25.729+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:25.752 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:25.747+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:25.796 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:25.786+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:25.904 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:25.895+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:26.462 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:26.454+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:26.521 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:26.512+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:26.583 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:26.574+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:26.707 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:26.699+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:26.739 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:26.734+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:26.815 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:26.809+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:26.877 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:26.861+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:27.489 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:27.480+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:27.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:27.501+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:27.541 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:27.532+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:27.718 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:27.710+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:27.749 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:27.740+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:27.840 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:27.831+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:27.859 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:27.853+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:28.471 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:28.461+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:28.492 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:28.479+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:28.555 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:28.545+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:28.695 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:28.680+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:28.790 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:28.783+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:28.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:28.822+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:28.850 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:28.843+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:29.447 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:29.439+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:29.524 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:29.514+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:29.574 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:29.566+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:29.658 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:29.650+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:29.798 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:29.789+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:29.799 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:29.792+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:29.886 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:29.878+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:30.486 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:30.478+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:30.497 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:30.489+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:30.560 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:30.552+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:30.620 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:30.613+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:30.768 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:30.761+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:30.782 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:30.775+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:30.866 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:30.859+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:31.494 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:31.487+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:31.497 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:31.489+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:31.533 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:31.521+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:31.583 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:31.577+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:31.772 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:31.764+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:31.801 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:31.794+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:31.869 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:31.861+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:32.490 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:32.482+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:32.535 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:32.528+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:32.537 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:32.525+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:32.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:32.538+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:32.758 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:32.752+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:32.787 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:32.779+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:32.843 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:32.835+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:33.505 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:33.497+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:33.515 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:33.507+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:33.550 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:33.541+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:33.562 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:33.555+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:33.770 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:33.763+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:33.784 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:33.776+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:33.811 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:33.803+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:34.492 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:34.485+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:34.537 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:34.522+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:34.570 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:34.562+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:34.600 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:34.591+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:34.778 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:34.761+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:34.804 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:34.796+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:34.804 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:34.796+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:35.525 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:35.518+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:35.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:35.538+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:35.565 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:35.548+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:35.645 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:35.633+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:35.732 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:35.726+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:35.777 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:35.771+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:35.837 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:35.820+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:36.490 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:39:36.483+0000 1694c700 -1 osd.6 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.610831+0000 front 2019-11-29T18:38:31.532118+0000 (oldest deadline 2019-11-29T18:39:15.234436+0000) 2019-11-29T18:39:36.553 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:36.548+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:36.576 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:36.570+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:36.636 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:36.629+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:36.704 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:36.696+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:36.785 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:36.777+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:36.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:36.779+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:36.993 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:26:49.881 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:36.998 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:26:49.887 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.007 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:26:49.895 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.017 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:26:49.898 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.017 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:26:49.782 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.022 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:26:49.795 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.124 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:26:50.012 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.143 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:26:50.030 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.178 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:26:49.950 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:39:37.585 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:37.576+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:37.601 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:37.595+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:37.691 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:37.683+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:37.708 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:37.702+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:37.799 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:37.790+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:37.809 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:37.801+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:38.539 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:38.532+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:38.653 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:38.646+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:38.654 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:38.647+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:38.729 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:38.720+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:38.789 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:38.782+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:38.862 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:38.854+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:39.554 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:39.546+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:39.672 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:39.663+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:39.676 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:39.669+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:39.737 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:39.732+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:39.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:39.745+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:39.820 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:39.812+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:40.545 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:40.537+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:40.634 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:40.626+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:40.651 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:40.646+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:40.694 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:40.688+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:40.722 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:40.717+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:40.803 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:40.795+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:41.552 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:41.545+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:41.589 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:41.582+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:41.671 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:41.665+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:41.688 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:41.682+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:41.724 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:41.715+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:41.812 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:41.804+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:42.513 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:42.505+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:42.592 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:42.584+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:42.644 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:42.637+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:42.689 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:42.683+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:42.735 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:42.729+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:42.781 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:42.769+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:43.502 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:43.493+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:43.580 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:43.570+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:43.614 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:43.604+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:43.682 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:43.676+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:43.707 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:43.698+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:43.808 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:43.801+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:44.512 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:44.504+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:44.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:44.597+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:44.629 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:44.622+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:44.724 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:44.719+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:44.756 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:44.747+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:44.836 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:44.828+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:45.470 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:39:45.466+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e488 v488) 2019-11-29T18:39:45.560 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:45.552+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:45.592 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:45.585+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:45.675 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:45.662+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:45.721 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:45.715+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:45.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:45.781+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:45.825 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:45.817+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:46.519 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:46.512+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:46.635 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:46.627+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:46.669 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:46.661+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:46.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:46.764+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:46.770 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:46.765+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:46.873 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:46.862+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:47.506 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:47.498+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:47.659 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:47.652+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:47.711 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:47.704+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:47.761 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:47.754+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:47.810 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:47.804+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:47.903 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:47.894+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:48.545 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:48.538+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:48.692 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:48.681+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:48.714 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:48.707+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:48.769 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:48.762+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:48.769 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:48.763+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:48.895 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:48.888+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:49.555 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:49.546+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:49.704 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:39:49.695+0000 1694c700 -1 osd.1 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.770303+0000 front 2019-11-29T18:38:30.015866+0000 (oldest deadline 2019-11-29T18:39:15.655837+0000) 2019-11-29T18:39:49.740 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:39:49.733+0000 1694c700 -1 osd.7 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:31.957345+0000 front 2019-11-29T18:38:31.923082+0000 (oldest deadline 2019-11-29T18:39:11.673512+0000) 2019-11-29T18:39:49.741 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:49.735+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:49.743 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:39:49.738+0000 1694c700 -1 osd.3 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:27.754102+0000 front 2019-11-29T18:38:28.181637+0000 (oldest deadline 2019-11-29T18:39:13.005859+0000) 2019-11-29T18:39:49.931 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:49.924+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:39:50.509 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:39:50.501+0000 1694c700 -1 osd.4 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:28.746346+0000 front 2019-11-29T18:38:31.827744+0000 (oldest deadline 2019-11-29T18:39:11.616283+0000) 2019-11-29T18:39:50.553 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:39:50.544+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 44sec e488 v488) 2019-11-29T18:39:50.763 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:39:50.756+0000 1694c700 -1 osd.2 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.630624+0000 front 2019-11-29T18:38:29.772956+0000 (oldest deadline 2019-11-29T18:39:11.686402+0000) 2019-11-29T18:39:50.919 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:39:50.912+0000 1694c700 -1 osd.5 488 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:38:29.197550+0000 front 2019-11-29T18:38:29.187095+0000 (oldest deadline 2019-11-29T18:39:13.829714+0000) 2019-11-29T18:40:22.694 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:22.493+0000 39a7c700 -1 osd.2 492 build_incremental_map_msg missing incremental map 492 2019-11-29T18:40:22.812 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:22.650+0000 37a78700 -1 osd.2 492 build_incremental_map_msg missing incremental map 492 2019-11-29T18:40:22.855 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:22.732+0000 34271700 -1 osd.2 492 build_incremental_map_msg missing incremental map 492 2019-11-29T18:40:22.887 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:22.743+0000 39a7c700 -1 osd.2 492 build_incremental_map_msg missing incremental map 492 2019-11-29T18:40:47.386 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:40:47.315+0000 20e22700 -1 RGWWatcher::handle_error cookie 363930816 err (107) Transport endpoint is not connected 2019-11-29T18:40:49.102 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:49.095+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:49.477 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:49.460+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:49.794 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:49.787+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:50.081 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:50.074+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:50.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:50.484+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:50.534 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:50.526+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:50.805 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:50.797+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:51.106 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:51.099+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:51.504 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:51.497+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:51.518 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:51.510+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:51.729 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:51.720+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:51.785 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:51.777+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:52.160 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:52.152+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:52.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:52.511+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:52.542 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:52.536+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:52.762 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:52.755+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:52.763 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:52.756+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:53.201 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:53.193+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:53.292 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:53.286+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:53.503 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:53.496+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:53.567 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:53.559+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:53.617 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:53.612+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:53.758 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:53.750+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:53.797 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:53.790+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:54.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:54.249+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:54.277 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:54.271+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:54.485 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:54.477+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:54.582 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:54.576+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:54.598 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:54.591+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:54.753 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:54.743+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:54.807 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:54.797+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:55.233 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:55.227+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:55.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:55.296+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:55.460 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:55.453+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:55.555 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:55.550+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:55.631 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:55.625+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:55.743 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:55.736+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:55.838 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:55.830+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:56.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:56.185+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:56.345 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:56.338+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:56.508 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:56.494+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:56.552 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:56.546+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:56.675 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:56.669+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:56.704 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:56.696+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:56.850 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:56.843+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:57.168 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:57.161+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:57.364 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:57.356+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:57.513 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:57.506+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:57.544 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:57.536+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:57.668 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:57.662+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:57.681 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:57.673+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:57.886 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:57.878+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:58.222 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:58.210+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:58.397 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:58.388+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:58.491 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:58.482+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:58.520 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:58.513+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:58.661 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:58.656+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:58.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:58.699+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:58.896 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:58.889+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:40:59.196 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:40:59.188+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:40:59.376 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:40:59.368+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:40:59.496 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:40:59.490+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:40:59.574 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:40:59.567+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:40:59.651 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:40:59.646+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:40:59.689 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:40:59.681+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:40:59.902 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:40:59.895+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:00.234 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:00.227+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:00.418 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:00.411+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:00.492 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:00.485+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:00.602 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:00.594+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:00.703 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:00.696+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:00.705 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:00.698+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:00.716 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:00.708+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:00.913 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:00.906+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:01.250 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:01.243+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:01.403 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:01.395+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:01.521 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:01.513+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:01.618 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:01.611+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:01.686 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:01.679+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:01.687 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:01.681+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:01.706 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:01.699+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:01.871 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:01.863+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:02.245 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:02.236+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:02.445 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:02.437+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:02.519 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:02.512+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:02.602 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:02.595+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:02.687 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:02.679+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:02.721 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:02.713+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:02.723 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:02.716+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:02.839 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:02.831+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:03.241 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:03.234+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:03.469 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:03.461+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:03.498 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:03.491+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:03.531 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:41:03.502+0000 20e22700 -1 RGWWatcher::handle_error cookie 5418469088 err (107) Transport endpoint is not connected 2019-11-29T18:41:03.596 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:03.588+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:03.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:03.661+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:03.718 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:03.711+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:03.719 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:03.714+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:03.822 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:03.815+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:04.226 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:04.220+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:04.518 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:04.510+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:04.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:04.518+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:04.626 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:04.620+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:04.682 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:04.675+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:04.770 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:04.759+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:04.772 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:04.763+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:04.824 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:04.818+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:05.199 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:05.192+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:05.520 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:05.512+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:05.557 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:05.550+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:05.590 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:05.582+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:05.674 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:05.666+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:05.721 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:05.714+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:05.721 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:05.716+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:05.799 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:05.792+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:06.174 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:06.166+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:06.517 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:06.510+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:06.597 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:06.590+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:06.603 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:06.595+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:06.724 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:06.716+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:06.759 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:06.749+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:06.760 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:06.751+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:06.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:06.809+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:07.135 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:07.127+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:07.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:07.475+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:07.571 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:07.562+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:07.595 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:07.588+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:07.687 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:07.679+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:07.783 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:07.775+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:07.799 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:07.792+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:07.801 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:07.795+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:08.142 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:08.135+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:08.525 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:08.519+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:08.526 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:08.521+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:08.594 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:08.586+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:08.601 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:08.593+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:08.727 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:08.720+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:08.761 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:08.754+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:08.815 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:08.809+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:08.816 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:08.811+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:09.139 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:09.133+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:09.517 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:09.511+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:09.518 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:09.513+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:09.573 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:09.565+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:09.625 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:09.616+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:09.700 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:09.692+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:09.716 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:09.709+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:09.866 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:09.860+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:09.867 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:09.862+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:10.100 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:10.094+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:10.514 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:10.509+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:10.516 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:10.512+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:10.551 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:10.543+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:10.584 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:10.577+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:10.701 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:10.693+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:10.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:10.698+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:10.899 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:10.894+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:10.902 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:10.897+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:11.145 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:11.139+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:11.524 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:11.519+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:11.524 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:11.520+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:11.564 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:11.557+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:11.564 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:11.558+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:11.672 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:11.665+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:11.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:11.703+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:11.883 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:11.878+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:11.884 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:11.880+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:12.105 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:12.099+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:12.520 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:12.513+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:12.520 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:12.516+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:12.528 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:12.521+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:12.603 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:12.595+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:12.687 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:12.674+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:12.694 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:12.685+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:12.871 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:12.866+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:12.872 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:12.868+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:13.157 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:13.152+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:13.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:13.474+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:13.483 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:13.477+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:13.507 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:13.499+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:13.598 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:13.592+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:13.658 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:13.650+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:13.722 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:13.714+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:13.926 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:13.920+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:13.927 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:13.922+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:14.165 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:14.158+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:14.446 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:14.441+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:14.447 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:14.443+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:14.475 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:14.467+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:14.615 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:14.607+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:14.678 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:14.668+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:14.721 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:14.712+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:14.933 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:14.926+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:14.934 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:14.929+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:15.148 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:15.142+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:15.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:15.416+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:15.424 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:15.418+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:15.436 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:15.427+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:15.611 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:15.604+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:15.701 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:15.692+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:15.764 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:15.756+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:15.962 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:15.957+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:15.969 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:15.965+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:16.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:16.144+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:16.397 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:16.389+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:16.458 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:16.452+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:16.460 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:16.456+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:16.613 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:16.604+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:16.695 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:16.687+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:16.778 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:16.770+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:16.966 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:16.958+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:16.967 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:16.962+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:17.140 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:17.135+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:17.415 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:17.410+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:17.416 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:17.412+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:17.432 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:17.423+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:17.580 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:17.571+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:17.696 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:17.687+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:17.817 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:17.808+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:17.936 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:17.927+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:17.936 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:17.931+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:18.179 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:18.173+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:18.394 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:18.389+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:18.397 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:18.393+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:18.424 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:18.416+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:18.579 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:18.571+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:18.721 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:18.712+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:18.776 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:18.767+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:18.912 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:18.904+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:18.913 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:18.908+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:19.151 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:19.146+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:19.402 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:19.396+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:19.405 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:19.401+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:19.444 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:19.435+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:19.580 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:19.572+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:19.722 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:19.713+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:19.777 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:19.769+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:19.879 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:19.872+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:19.880 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:19.875+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:20.113 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:20.106+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:20.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:20.378+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:20.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:20.380+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:20.411 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:20.403+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:20.606 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:20.598+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:20.696 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:20.687+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:20.759 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:20.750+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:20.834 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:20.828+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:20.834 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:20.830+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:21.086 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:21.081+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:21.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:21.375+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:21.382 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:21.377+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:21.446 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:21.438+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:21.628 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:21.619+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:21.729 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:21.719+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:21.747 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:21.739+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:21.856 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:21.851+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:21.857 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:21.852+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:22.048 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:22.041+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:22.383 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:22.376+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:22.384 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:22.379+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:22.460 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:22.452+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:22.676 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:22.668+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:22.752 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:22.744+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:22.764 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:22.757+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:22.870 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:22.865+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:22.873 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:22.869+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:23.058 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:23.053+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:23.386 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:23.380+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:23.387 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:23.382+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:23.483 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:23.476+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:23.656 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:23.648+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:23.720 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:23.713+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:23.743 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:23.735+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:23.910 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:23.904+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:23.911 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:23.906+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:24.031 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:24.026+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:24.436 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:24.429+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:24.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:24.432+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:24.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:24.480+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:24.642 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:24.635+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:24.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:24.703+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:24.761 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:24.754+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:24.888 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:24.882+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:24.889 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:24.884+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:24.988 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:24.983+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:25.461 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:25.455+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:25.462 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:25.458+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:25.529 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:25.522+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:25.647 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:25.640+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:25.710 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:25.703+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:25.791 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:25.784+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:25.795 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:41:25.787+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e493 v493) 2019-11-29T18:41:25.844 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:25.839+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:25.847 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:25.843+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:25.982 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:25.977+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:26.437 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:26.431+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:26.440 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:26.435+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:26.566 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:26.559+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:26.660 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:26.652+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:26.757 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:26.749+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:26.790 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:26.783+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:26.806 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:26.801+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:26.809 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:26.804+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:26.948 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:26.943+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:27.480 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:27.474+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:27.482 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:27.477+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:27.584 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:27.577+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:27.714 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:27.706+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:27.748 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:27.740+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:27.767 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:27.759+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:27.851 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:27.845+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:27.851 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:27.846+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:27.968 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:27.963+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:28.515 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:28.508+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:28.515 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:28.510+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:28.615 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:28.607+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:28.696 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:28.689+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:28.732 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:28.724+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:28.762 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:28.754+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:28.826 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:28.820+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:28.827 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:28.822+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:28.993 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:28.988+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:29.552 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:29.548+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:29.554 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:29.550+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:29.634 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:29.626+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:29.699 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:29.691+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:29.759 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:29.751+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:29.777 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:29.769+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:29.808 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:29.800+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:29.809 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:29.804+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:29.966 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:29.958+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:30.578 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:30.572+0000 1694c700 -1 osd.3 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.639381+0000 front 2019-11-29T18:40:05.646198+0000 (oldest deadline 2019-11-29T18:40:48.449925+0000) 2019-11-29T18:41:30.579 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:30.574+0000 1694c700 -1 osd.3 493 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:30.667 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:41:30.660+0000 1694c700 -1 osd.7 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:06.031474+0000 front 2019-11-29T18:40:06.180869+0000 (oldest deadline 2019-11-29T18:40:50.109987+0000) 2019-11-29T18:41:30.673 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:41:30.666+0000 1694c700 -1 osd.6 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:04.203652+0000 front 2019-11-29T18:40:04.168688+0000 (oldest deadline 2019-11-29T18:40:48.856533+0000) 2019-11-29T18:41:30.753 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:41:30.746+0000 1694c700 -1 osd.4 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:03.460839+0000 front 2019-11-29T18:40:04.075482+0000 (oldest deadline 2019-11-29T18:40:49.261621+0000) 2019-11-29T18:41:30.781 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:41:30.774+0000 1694c700 -1 osd.5 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:05.964625+0000 front 2019-11-29T18:40:05.982928+0000 (oldest deadline 2019-11-29T18:40:51.235260+0000) 2019-11-29T18:41:30.814 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:30.807+0000 1694c700 -1 osd.0 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:40:20.097993+0000 (oldest deadline 2019-11-29T18:41:00.097993+0000) 2019-11-29T18:41:30.818 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:41:30.814+0000 1694c700 -1 osd.0 493 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:111818 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e492) 2019-11-29T18:41:30.829 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:41:30.820+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e493 v493) 2019-11-29T18:41:30.937 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:41:30.931+0000 1694c700 -1 osd.2 493 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:40:07.097338+0000 front 2019-11-29T18:40:07.120260+0000 (oldest deadline 2019-11-29T18:40:52.378383+0000) 2019-11-29T18:41:31.655 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:31.651+0000 1694c700 -1 osd.3 494 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:41:32.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:41:32.703+0000 1694c700 -1 osd.3 495 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:111839 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5418469088 gen 1] snapc 0=[] ondisk+write+known_if_redirected e493) 2019-11-29T18:42:29.986 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:42:29.561+0000 1694c700 -1 osd.0 499 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:111935 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e496) 2019-11-29T18:43:23.492 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:23.485+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:24.548 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:24.539+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:24.567 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:24.562+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:24.568 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:24.560+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:24.713 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:43:24.704+0000 1694c700 -1 osd.6 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.096099+0000 front 2019-11-29T18:42:38.624137+0000 (oldest deadline 2019-11-29T18:43:23.760537+0000) 2019-11-29T18:43:24.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:24.889+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:25.548 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:25.542+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:25.555 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:25.547+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:25.578 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:25.573+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:25.890 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:25.883+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:26.316 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:26.310+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:26.417 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:26.409+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:26.542 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:26.534+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:26.550 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:26.544+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:26.561 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:26.555+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:26.918 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:26.910+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:27.371 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:27.365+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:27.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:27.379+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:27.508 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:27.501+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:27.527 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:27.516+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:27.545 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:27.524+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:27.965 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:27.948+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:28.362 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:28.355+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:28.385 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:28.379+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:28.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:28.475+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:28.552 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:28.545+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:28.582 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:28.575+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:28.930 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:28.923+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:29.322 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:29.315+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:29.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:29.413+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:29.443 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:29.435+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:29.543 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:29.537+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:29.557 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:29.550+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:29.929 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:29.918+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:30.381 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:30.366+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:30.430 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:30.422+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:30.451 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:30.444+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:30.517 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:30.510+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:30.564 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:30.558+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:30.883 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:30.875+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:31.370 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:31.361+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:31.394 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:31.386+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:31.425 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:31.420+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:31.541 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:31.534+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:31.600 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:31.593+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:31.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:31.834+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:32.371 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:32.363+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:32.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:32.381+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:32.460 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:32.394+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:32.549 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:32.543+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:32.625 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:32.618+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:32.827 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:32.820+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:33.351 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:33.343+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:33.399 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:33.389+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:33.400 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:33.389+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:33.551 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:33.545+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:33.628 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:33.623+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:33.831 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:33.823+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:34.360 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:34.352+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:34.395 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:34.389+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:34.400 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:34.393+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:34.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:34.582+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:34.593 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:34.588+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:34.827 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:34.820+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:35.355 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:35.349+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:35.395 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:35.388+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:35.444 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:35.437+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:35.573 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:35.568+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:35.634 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:35.627+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:35.873 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:35.867+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:36.336 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:30:49.224 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.344 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:30:49.233 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.357 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:30:49.128 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.359 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:30:49.241 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.361 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:30:49.243 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.365 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:30:49.138 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.388 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:36.379+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:36.398 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:36.392+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:36.450 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:36.443+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:36.565 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:36.559+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:36.580 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:30:49.468 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.600 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:36.593+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:36.683 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:30:49.456 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.702 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:30:49.589 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:43:36.900 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:36.894+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:37.373 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:37.368+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:37.412 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:37.401+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:37.458 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:37.451+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:37.560 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:37.554+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:37.563 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:37.558+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:37.876 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:37.869+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:38.375 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:38.370+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:38.402 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:38.393+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:38.450 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:38.442+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:38.548 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:38.541+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:38.611 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:38.605+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:38.845 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:38.835+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:39.423 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:39.418+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:39.426 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:39.413+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:39.461 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:39.453+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:39.513 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:39.506+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:39.639 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:39.633+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:39.894 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:39.886+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:40.380 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:40.373+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:40.465 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:40.458+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:40.467 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:40.459+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:40.521 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:40.515+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:40.653 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:40.648+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:40.877 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:40.869+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:41.410 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:41.402+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:41.441 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:41.433+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:41.520 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:41.510+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:41.549 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:41.532+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:41.623 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:41.616+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:41.866 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:41.859+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:42.446 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:42.439+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:42.483 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:42.471+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:42.512 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:42.499+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:42.523 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:42.517+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:42.677 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:42.664+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:42.917 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:42.909+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:43.403 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:43.397+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:43.485 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:43.477+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:43.516 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:43.508+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:43.547 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:43.541+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:43.653 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:43.647+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:43.908 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:43.899+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:44.439 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:44.433+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:44.456 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:44.441+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:44.557 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:44.550+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:44.578 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:44.572+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:44.673 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:44.666+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:44.936 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:44.929+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:45.459 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:45.452+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:45.467 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:45.459+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:45.521 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:45.513+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:45.544 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:45.536+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:45.649 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:45.643+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:45.959 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:45.950+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:46.462 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:46.453+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:46.465 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:46.458+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:46.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:46.540+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:46.580 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:46.574+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:46.623 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:46.617+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:46.993 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:46.984+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:47.425 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:47.417+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:47.456 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:47.450+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:47.562 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:47.554+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:47.578 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:47.573+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:47.602 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:47.596+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:48.030 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:48.021+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:48.383 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:48.374+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:48.476 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:48.471+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:48.549 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:48.540+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:48.559 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:48.554+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:48.580 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:48.575+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:49.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:48.995+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:49.391 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:49.381+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:49.503 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:49.497+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:49.519 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:49.508+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:49.532 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:49.523+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:49.588 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:49.581+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:49.991 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:49.982+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:50.430 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:50.410+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:50.488 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:50.479+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:50.509 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:50.501+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:50.547 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:50.541+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:50.576 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:50.569+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:50.990 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:50.981+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:51.434 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:51.425+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:51.463 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:51.456+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:51.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:51.501+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:51.553 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:51.546+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:51.613 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:51.607+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:52.029 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:52.021+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:52.475 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:52.466+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:52.494 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:52.487+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:52.501 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:52.494+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:52.556 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:52.536+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:52.584 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:52.577+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:53.033 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:53.026+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:53.441 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:53.433+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:53.474 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:53.465+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:53.517 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:53.510+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:53.518 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:53.511+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:53.606 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:53.600+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:54.087 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:54.069+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:54.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:54.407+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:54.458 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:54.449+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:54.573 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:54.557+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:54.573 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:54.560+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:54.576 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:54.569+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:55.067 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:55.059+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:55.393 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:55.385+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:55.510 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:55.502+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:55.527 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:55.521+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:43:55.603 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:43:55.596+0000 1694c700 -1 osd.1 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.842785+0000 front 2019-11-29T18:42:40.845419+0000 (oldest deadline 2019-11-29T18:43:23.658710+0000) 2019-11-29T18:43:55.622 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:43:55.615+0000 1694c700 -1 osd.2 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.823390+0000 front 2019-11-29T18:42:40.256607+0000 (oldest deadline 2019-11-29T18:43:22.794536+0000) 2019-11-29T18:43:56.064 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:43:56.057+0000 1694c700 -1 osd.5 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.273418+0000 front 2019-11-29T18:42:40.255669+0000 (oldest deadline 2019-11-29T18:43:24.485668+0000) 2019-11-29T18:43:56.308 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:43:56.302+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 43sec e499 v499) 2019-11-29T18:43:56.359 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:43:56.351+0000 1694c700 -1 osd.4 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:39.004723+0000 front 2019-11-29T18:42:37.849648+0000 (oldest deadline 2019-11-29T18:43:23.626251+0000) 2019-11-29T18:43:56.492 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:43:56.486+0000 1694c700 -1 osd.7 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:41.132581+0000 front 2019-11-29T18:42:41.079191+0000 (oldest deadline 2019-11-29T18:43:25.665060+0000) 2019-11-29T18:43:56.557 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:43:56.548+0000 1694c700 -1 osd.3 499 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:42:40.779153+0000 front 2019-11-29T18:42:40.767394+0000 (oldest deadline 2019-11-29T18:43:25.992848+0000) 2019-11-29T18:44:28.229 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:28.154+0000 38e7b700 -1 osd.4 502 build_incremental_map_msg missing incremental map 502 2019-11-29T18:44:28.260 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:28.249+0000 36e77700 -1 osd.4 502 build_incremental_map_msg missing incremental map 502 2019-11-29T18:44:28.327 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:28.268+0000 33670700 -1 osd.4 502 build_incremental_map_msg missing incremental map 502 2019-11-29T18:44:28.531 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:28.510+0000 39e7d700 -1 osd.4 502 build_incremental_map_msg missing incremental map 502 2019-11-29T18:44:54.684 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:54.676+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:54.763 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:54.756+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:55.654 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:55.647+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:55.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:55.720+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:56.685 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:56.678+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:56.708 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:56.700+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:44:56.763 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:56.756+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:56.920 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:44:56.912+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:44:57.328 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:44:57.322+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:44:57.659 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:57.652+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:57.745 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:57.737+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:44:57.751 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:57.744+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:57.931 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:44:57.923+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:44:58.040 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:44:58.005+0000 20e22700 -1 RGWWatcher::handle_error cookie 5432189456 err (107) Transport endpoint is not connected 2019-11-29T18:44:58.343 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:44:58.337+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:44:58.696 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:58.688+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:58.711 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:58.704+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:44:58.728 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:58.723+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:58.928 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:44:58.921+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:44:58.978 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:44:58.973+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:44:59.333 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:44:59.327+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:44:59.722 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:44:59.717+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:44:59.801 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:44:59.723+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:44:59.801 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:44:59.743+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:44:59.909 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:44:59.902+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:44:59.998 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:44:59.993+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:00.356 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:00.350+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:00.688 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:00.682+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:00.781 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:00.773+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:00.804 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:00.796+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:00.911 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:00.904+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:00.963 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:00.955+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:01.289 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:01.280+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:01.401 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:01.393+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:01.729 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:01.723+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:01.768 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:01.761+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:01.812 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:01.804+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:01.952 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:01.945+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:01.974 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:01.970+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:02.307 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:02.298+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:02.407 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:02.398+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:02.728 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:02.722+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:02.779 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:02.771+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:02.784 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:02.775+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:02.946 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:02.937+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:02.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:02.993+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:03.325 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:03.300+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:03.443 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:03.433+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:03.748 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:03.739+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:03.776 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:03.770+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:03.778 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:03.770+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:03.996 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:03.988+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:04.024 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:04.013+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:04.291 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:04.283+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:04.489 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:04.483+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:04.778 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:04.769+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:04.786 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:04.778+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:04.829 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:04.823+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:05.035 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:05.031+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:05.043 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:05.035+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:05.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:05.314+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:05.499 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:05.493+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:05.742 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:05.734+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:05.769 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:05.761+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:05.782 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:05.777+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:06.063 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:06.054+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:06.074 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:06.066+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:06.075 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:06.070+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:06.346 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:06.338+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:06.508 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:06.502+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:06.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:06.697+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:06.802 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:06.797+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:06.818 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:06.809+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:07.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:07.049+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:07.063 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:07.057+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:07.064 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:07.060+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:07.305 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:07.296+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:07.542 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:07.536+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:07.733 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:07.725+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:07.787 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:07.779+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:07.850 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:07.844+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:08.112 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:08.103+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:08.112 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:08.106+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:08.116 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:08.111+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:08.290 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:08.281+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:08.585 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:08.579+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:08.709 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:08.700+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:08.762 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:08.753+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:08.839 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:08.831+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:09.115 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:09.108+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:09.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:09.111+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:09.139 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:09.130+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:09.292 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:09.283+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:09.623 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:09.618+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:09.680 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:09.669+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:09.801 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:09.793+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:09.824 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:09.818+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:10.082 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:10.073+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:10.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:10.078+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:10.188 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:10.178+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:10.289 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:10.281+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:10.637 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:10.628+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:10.650 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:10.644+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:10.813 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:10.805+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:10.848 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:10.843+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:11.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:11.094+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:11.106 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:11.100+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:11.175 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:11.166+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:11.247 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:11.238+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:11.597 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:11.587+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:11.672 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:11.660+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:11.795 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:11.786+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:11.854 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:11.848+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:12.136 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:12.131+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:12.139 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:12.134+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:12.147 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:12.140+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:12.301 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:12.293+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:12.564 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:12.557+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:12.666 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:12.660+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:12.801 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:12.793+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:12.877 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:12.872+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:13.116 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:13.110+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:13.120 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:13.115+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:13.163 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:13.155+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:13.303 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:13.292+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:13.586 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:13.578+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:13.673 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:13.667+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:13.779 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:13.772+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:13.921 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:13.914+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:14.077 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:14.072+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:14.078 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:14.073+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:14.117 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:14.111+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:14.263 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:14.255+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:14.631 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:14.624+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:14.647 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:14.641+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:14.818 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:14.808+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:14.881 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:14.877+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:15.081 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:15.073+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:15.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:15.124+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:15.132 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:15.126+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:15.278 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:15.270+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:15.623 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:15.617+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:15.637 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:15.629+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:15.850 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:15.843+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:15.908 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:15.900+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:16.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:16.050+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:16.161 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:16.155+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:16.164 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:16.159+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:16.249 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:16.242+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:16.584 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:16.579+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:16.659 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:16.651+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:16.819 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:16.805+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:16.865 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:16.855+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:17.046 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:17.038+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:17.117 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:17.112+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:17.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:17.113+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:17.301 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:17.294+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:17.614 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:17.608+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:17.682 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:17.674+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:17.786 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:17.779+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:17.898 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:17.892+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:18.010 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:18.002+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:18.127 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:18.119+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:18.129 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:18.123+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:18.291 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:18.283+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:18.648 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:18.642+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:18.656 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:18.648+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:18.785 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:18.778+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:18.919 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:18.914+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:19.021 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:19.014+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:19.164 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:19.159+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:19.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:19.161+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:19.277 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:19.269+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:19.692 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:19.686+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:19.709 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:19.701+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:19.778 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:19.768+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:19.914 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:19.909+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:20.025 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:20.014+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:20.198 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:20.190+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:20.199 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:20.194+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:20.235 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:20.228+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:20.735 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:20.727+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:20.740 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:20.735+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:20.754 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:20.748+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:20.915 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:20.909+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:21.072 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:21.064+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:21.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:21.167+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:21.174 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:21.168+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:21.274 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:21.266+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:21.705 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:21.696+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:21.733 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:21.724+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:21.745 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:21.737+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:21.955 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:21.949+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:22.076 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:22.069+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:22.130 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:22.123+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:22.134 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:22.129+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:22.308 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:22.301+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:22.703 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:22.693+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:22.717 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:22.709+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:22.739 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:22.731+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:22.996 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:22.989+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:23.050 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:23.041+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:23.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:23.154+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:23.165 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:23.156+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:23.273 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:23.265+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:23.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:23.679+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:23.731 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:23.723+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:23.738 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:23.731+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:24.034 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:24.027+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:24.042 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:24.034+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:24.164 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:24.150+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:24.166 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:24.156+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:24.299 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:24.292+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:24.691 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:24.684+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:24.781 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:24.768+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:24.781 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:24.774+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:25.036 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:25.030+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:25.057 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:25.050+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:25.199 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:25.193+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:25.202 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:25.196+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:25.269 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:25.262+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:25.678 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:25.671+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:25.759 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:25.752+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:25.794 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:25.787+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:25.933 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:45:25.682+0000 20e22700 -1 RGWWatcher::handle_error cookie 363716656 err (107) Transport endpoint is not connected 2019-11-29T18:45:26.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:26.054+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:26.109 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:26.101+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:26.158 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:26.150+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:26.160 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:26.153+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:26.299 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:26.292+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:26.548 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:45:26.540+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e503 v503) 2019-11-29T18:45:26.692 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:26.686+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:26.717 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:26.710+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:26.773 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:26.766+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:27.044 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:27.038+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:27.113 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:27.105+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:27.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:27.187+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:27.196 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:27.189+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:27.286 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:27.279+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:27.668 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:27.661+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:27.678 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:27.672+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:27.771 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:27.764+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:28.098 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:28.091+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:28.118 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:28.111+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:28.171 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:28.164+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:28.173 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:28.167+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:28.324 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:28.317+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:28.661 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:28.654+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:28.700 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:28.693+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:28.766 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:28.759+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:29.132 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:29.126+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:29.133 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:29.127+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:29.145 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:29.137+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:29.145 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:29.138+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:29.146 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:29.141+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:29.287 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:29.280+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:29.637 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:29.629+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:29.691 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:29.684+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:29.813 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:29.806+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:30.120 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:30.111+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:30.126 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:30.119+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:30.127 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:30.122+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:30.176 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:30.166+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:30.176 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:30.170+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:30.303 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:30.296+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:30.646 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:30.639+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:30.744 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:30.736+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:30.817 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:30.809+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:31.093 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:31.086+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:31.115 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:31.110+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:31.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:31.112+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:31.212 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:31.206+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:31.213 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:31.208+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:31.318 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:31.311+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:31.563 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:45:31.553+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e503 v503) 2019-11-29T18:45:31.615 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:31.603+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:31.713 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:31.706+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:31.833 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:31.826+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:32.054 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:32.046+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:32.121 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:32.116+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:32.122 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:32.117+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:32.196 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:32.188+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:32.197 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:32.191+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:32.302 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:32.295+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:32.605 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:32.597+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:32.747 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:32.740+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:32.830 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:32.823+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:33.058 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:33.051+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:33.119 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:33.114+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:33.121 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:33.116+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:33.151 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:33.144+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:33.151 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:33.146+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:33.321 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:33.313+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:33.605 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:33.598+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:33.765 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:33.758+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:33.878 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:33.871+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:34.023 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:34.016+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:34.123 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:34.117+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:34.125 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:34.120+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:34.179 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:34.172+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:34.180 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:34.174+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:34.315 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:34.308+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:34.583 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:34.575+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:34.752 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:34.746+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:34.930 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:34.923+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:35.090 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:35.084+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:35.091 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:35.085+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:35.100 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:35.090+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:35.147 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:35.141+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:35.148 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:35.142+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:35.309 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:35.302+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:35.628 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:35.621+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:35.797 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:35.784+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:35.944 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:45:35.937+0000 1694c700 -1 osd.4 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.708692+0000 front 2019-11-29T18:44:13.895907+0000 (oldest deadline 2019-11-29T18:44:55.988966+0000) 2019-11-29T18:45:36.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:36.040+0000 1694c700 -1 osd.3 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:13.191534+0000 front 2019-11-29T18:44:13.229590+0000 (oldest deadline 2019-11-29T18:44:54.203476+0000) 2019-11-29T18:45:36.058 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:36.043+0000 1694c700 -1 osd.3 503 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:36.142 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:45:36.135+0000 1694c700 -1 osd.7 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:11.163521+0000 front 2019-11-29T18:44:11.165743+0000 (oldest deadline 2019-11-29T18:44:56.754904+0000) 2019-11-29T18:45:36.145 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:36.139+0000 1694c700 -1 osd.0 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:44:25.283473+0000 (oldest deadline 2019-11-29T18:45:05.283473+0000) 2019-11-29T18:45:36.146 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:36.140+0000 1694c700 -1 osd.0 503 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4465.0:112197 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e502) 2019-11-29T18:45:36.350 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:45:36.343+0000 1694c700 -1 osd.6 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:09.099886+0000 front 2019-11-29T18:44:09.106828+0000 (oldest deadline 2019-11-29T18:45:00.460838+0000) 2019-11-29T18:45:36.615 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:45:36.604+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 43sec e503 v503) 2019-11-29T18:45:36.645 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:45:36.637+0000 1694c700 -1 osd.5 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.532090+0000 front 2019-11-29T18:44:12.534106+0000 (oldest deadline 2019-11-29T18:44:53.928395+0000) 2019-11-29T18:45:36.771 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:45:36.765+0000 1694c700 -1 osd.2 503 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:44:12.328241+0000 front 2019-11-29T18:44:12.289822+0000 (oldest deadline 2019-11-29T18:44:56.323453+0000) 2019-11-29T18:45:37.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:37.174+0000 1694c700 -1 osd.3 504 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:37.197 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:45:37.192+0000 1694c700 -1 osd.0 504 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4465.0:112208 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e503) 2019-11-29T18:45:38.157 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:45:38.151+0000 1694c700 -1 osd.3 505 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112250 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 363716656] snapc 0=[] ondisk+write+known_if_redirected e503) 2019-11-29T18:45:46.801 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:32:59.690 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.808 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:32:59.696 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.852 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:32:59.625 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.856 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:32:59.629 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.869 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:32:59.757 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.872 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:32:59.760 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.964 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:32:59.853 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:46.990 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:32:59.763 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:45:47.038 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:32:59.926 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:07.786 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:07.778+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:08.795 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:08.787+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:08.797 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:08.789+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:09.151 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:09.144+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:09.160 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:09.152+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:09.622 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:09.614+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:09.750 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:09.744+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:09.816 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:09.808+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:10.147 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:10.140+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:10.165 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:10.158+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:10.673 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:10.666+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:10.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:10.740+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:10.823 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:10.814+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:10.947 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:10.939+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:10.969 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:10.964+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:11.132 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:11.127+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:11.176 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:11.168+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:11.696 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:11.688+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:11.726 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:11.721+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:11.865 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:11.857+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:11.923 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:11.916+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:12.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:12.005+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:12.132 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:12.127+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:12.165 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:12.158+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:12.688 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:12.680+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:12.730 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:12.724+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:12.868 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:12.860+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:12.915 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:12.906+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:12.998 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:12.990+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:13.118 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:13.111+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:13.183 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:13.176+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:13.669 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:13.662+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:13.767 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:13.760+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:13.904 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:13.897+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:13.926 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:13.919+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:14.036 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:14.029+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:14.089 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:14.083+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:14.222 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:14.215+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:14.702 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:14.695+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:14.763 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:14.757+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:14.921 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:14.914+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:14.923 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:14.917+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:15.051 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:15.044+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:15.134 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:15.129+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:15.199 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:15.191+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:15.746 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:15.739+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:15.813 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:15.806+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:15.942 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:15.934+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:15.963 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:15.955+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:16.067 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:16.062+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:16.172 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:16.166+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:16.179 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:16.171+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:16.708 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:16.701+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:16.821 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:16.815+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:16.932 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:16.924+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:16.956 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:16.949+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:17.045 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:17.040+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:17.146 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:17.142+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:17.205 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:17.198+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:17.765 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:17.757+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:17.816 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:17.811+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:17.942 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:17.933+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:17.975 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:17.968+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:18.037 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:18.033+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:18.179 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:18.172+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:18.232 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:18.225+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:18.748 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:18.741+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:18.805 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:18.799+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:18.932 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:18.923+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:18.962 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:18.954+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:19.075 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:19.070+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:19.213 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:19.208+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:19.220 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:19.212+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:19.772 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:19.764+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:19.773 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:19.768+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:19.982 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:19.974+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:19.986 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:19.979+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:20.055 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:20.048+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:20.182 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:20.176+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:20.237 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:20.231+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:20.807 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:20.800+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:20.810 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:20.804+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:20.945 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:20.937+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:21.028 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:21.022+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:21.035 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:21.028+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:21.228 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:21.221+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:21.257 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:21.249+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:21.786 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:21.781+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:21.827 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:21.820+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:21.955 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:21.947+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:22.012 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:22.004+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:22.045 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:22.040+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:22.193 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:22.186+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:22.270 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:22.262+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:22.827 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:22.819+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:22.829 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:22.823+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:22.994 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:22.986+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:23.012 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:23.007+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:23.055 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:23.047+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:23.156 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:23.150+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:23.324 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:23.316+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:23.826 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:23.819+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:23.873 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:23.864+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:23.979 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:23.972+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:24.010 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:24.004+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:24.113 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:24.108+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:24.115 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:24.108+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:24.287 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:24.279+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:24.795 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:24.789+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:24.915 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:24.909+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:24.973 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:24.966+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:24.995 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:24.988+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:25.092 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:25.085+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:25.162 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:25.156+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:25.338 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:25.331+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:25.831 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:25.823+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:25.935 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:25.929+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:25.978 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:25.972+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:25.997 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:25.990+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:26.122 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:26.115+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:26.199 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:26.192+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:26.316 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:26.308+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:26.807 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:26.800+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:26.923 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:26.917+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:26.950 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:26.945+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:26.981 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:26.974+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:27.128 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:27.121+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:27.157 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:27.151+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:27.343 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:27.336+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:27.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:27.757+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:27.929 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:27.921+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:27.958 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:27.950+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:27.989 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:27.984+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:28.094 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:28.086+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:28.156 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:28.151+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:28.337 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:28.330+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:28.767 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:28.760+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:28.945 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:28.939+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:28.953 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:28.947+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:28.988 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:28.980+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:29.112 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:29.105+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:29.144 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:29.138+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:29.305 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:29.297+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:29.784 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:29.776+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:29.912 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:29.907+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:29.948 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:29.941+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:29.989 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:29.984+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:30.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:30.089+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:30.170 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:30.165+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:30.339 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:30.332+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:30.750 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:30.743+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:30.875 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:30.870+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:30.918 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:30.910+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:30.984 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:30.978+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:31.137 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:31.131+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:31.145 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:31.139+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:31.315 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:31.307+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:31.722 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:31.714+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:31.876 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:31.867+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:31.924 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:31.918+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:32.004 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:31.997+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:32.161 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:32.156+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:32.165 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:34:45.054 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.172 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:34:45.061 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.185 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:34:45.069 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.185 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:34:45.073 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.189 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:32.181+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:32.198 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:34:44.970 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.215 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:34:44.988 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.291 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:32.282+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:32.321 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:34:45.188 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.338 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:34:45.225 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.420 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:34:45.193 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:47:32.684 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:32.677+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:32.905 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:32.897+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:32.971 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:32.967+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:33.028 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:33.024+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:33.124 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:33.117+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:33.192 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:33.186+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:33.324 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:33.318+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:33.713 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:33.706+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:33.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:33.913+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:33.997 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:33.992+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:34.027 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:34.022+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:34.137 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:34.130+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:34.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:34.158+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:34.361 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:34.352+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:34.764 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:34.756+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:34.961 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:34.953+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:35.017 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:35.012+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:35.028 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:35.022+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:35.161 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:35.155+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:35.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:35.177+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:35.414 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:35.407+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:35.749 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:35.740+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:35.933 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:35.925+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:35.996 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:35.990+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:36.059 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:36.052+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:36.117 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:36.112+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:36.149 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:36.133+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:36.426 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:36.418+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:36.754 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:36.745+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:36.952 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:36.943+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:36.966 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:36.960+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:37.033 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:37.026+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:37.124 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:37.118+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:37.158 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:37.149+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:37.387 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:37.379+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:37.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:37.715+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:37.956 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:37.947+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:38.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:38.005+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:38.065 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:38.058+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:38.116 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:38.108+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:38.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:38.182+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:38.425 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:38.417+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:38.759 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:38.749+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:38.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:38.990+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:39.000 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:38.993+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:39.019 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:39.012+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:39.127 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:39.121+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:39.199 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:39.191+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:39.434 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:39.426+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:39.752 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:39.744+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:39.970 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:39.962+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:40.001 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:39.993+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:40.061 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:40.054+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:40.099 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:40.092+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:40.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:40.176+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:40.407 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:40.399+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:40.798 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:40.790+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:40.964 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:40.957+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:41.015 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:41.007+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:41.027 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:41.021+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:41.140 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:41.134+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:41.184 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:41.172+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:41.424 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:41.415+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:47:41.779 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:47:41.771+0000 1694c700 -1 osd.7 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.141315+0000 front 2019-11-29T18:46:26.117707+0000 (oldest deadline 2019-11-29T18:47:09.573030+0000) 2019-11-29T18:47:41.953 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:47:41.946+0000 1694c700 -1 osd.3 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:23.164925+0000 front 2019-11-29T18:46:23.271117+0000 (oldest deadline 2019-11-29T18:47:08.715899+0000) 2019-11-29T18:47:41.989 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:47:41.981+0000 1694c700 -1 osd.4 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:27.416115+0000 front 2019-11-29T18:46:24.597540+0000 (oldest deadline 2019-11-29T18:47:07.385212+0000) 2019-11-29T18:47:42.042 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:47:42.035+0000 1694c700 -1 osd.2 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.261625+0000 front 2019-11-29T18:46:25.298770+0000 (oldest deadline 2019-11-29T18:47:10.416336+0000) 2019-11-29T18:47:42.156 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:47:42.148+0000 1694c700 -1 osd.6 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.934714+0000 front 2019-11-29T18:46:25.953232+0000 (oldest deadline 2019-11-29T18:47:09.978266+0000) 2019-11-29T18:47:42.181 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:47:42.174+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 42sec e508 v508) 2019-11-29T18:47:42.182 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:47:42.176+0000 1694c700 -1 osd.1 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:26.557102+0000 front 2019-11-29T18:46:26.808505+0000 (oldest deadline 2019-11-29T18:47:08.226732+0000) 2019-11-29T18:47:42.388 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:47:42.381+0000 1694c700 -1 osd.5 508 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:46:25.238671+0000 front 2019-11-29T18:46:24.879983+0000 (oldest deadline 2019-11-29T18:47:08.755023+0000) 2019-11-29T18:48:11.252 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:11.242+0000 28259700 -1 reset not still connected to 0xed4ded0 2019-11-29T18:48:11.458 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:11.242+0000 28259700 -1 reset not still connected to 0x3baa2f30 2019-11-29T18:48:39.787 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:48:39.764+0000 20e22700 -1 RGWWatcher::handle_error cookie 5452361024 err (107) Transport endpoint is not connected 2019-11-29T18:48:40.723 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:40.715+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:41.596 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:41.587+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:41.701 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:41.695+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:41.750 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:41.742+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:42.021 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:42.014+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:42.604 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:42.597+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:42.612 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:35:55.501 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.617 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:35:55.506 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.690 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:42.683+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:42.701 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:35:55.589 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.714 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:35:55.601 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.727 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:35:55.500 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.742 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:35:55.629 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.753 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:35:55.526 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.753 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:42.746+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:42.868 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:35:55.755 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:42.930 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:35:55.703 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:48:43.015 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:43.008+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:43.399 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:43.393+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:43.603 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:43.595+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:43.740 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:43.735+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:43.779 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:43.772+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:44.041 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:44.034+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:44.361 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:44.356+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:44.647 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:44.640+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:44.727 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:44.721+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:44.742 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:44.736+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:45.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:45.006+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:45.165 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:45.158+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:45.326 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:45.319+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:45.616 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:45.609+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:45.738 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:45.731+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:45.778 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:45.771+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:46.068 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:46.062+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:46.201 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:46.191+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:46.299 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:46.294+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:46.646 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:46.639+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:46.747 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:46.741+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:46.765 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:46.756+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:46.904 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:46.899+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:47.126 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:47.116+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:47.165 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:47.158+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:47.296 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:47.291+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:47.696 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:47.688+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:47.787 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:47.782+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:47.809 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:47.802+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:47.944 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:47.936+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:48.094 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:48.086+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:48.189 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:48.182+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:48.277 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:48.270+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:48.727 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:48.721+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:48.817 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:48.812+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:48.830 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:48.823+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:48.971 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:48.966+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:49.130 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:49.123+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:49.191 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:49.183+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:49.258 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:49.253+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:49.751 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:49.744+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:49.824 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:49.819+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:49.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:49.830+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:49.967 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:49.962+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:50.122 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:50.114+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:50.242 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:50.235+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:50.258 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:50.252+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:50.773 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:50.766+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:50.794 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:50.788+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:50.875 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:50.870+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:50.964 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:50.959+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:51.176 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:51.169+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:51.294 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:51.287+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:51.310 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:51.305+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:51.767 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:51.759+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:51.770 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:51.759+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:51.880 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:51.871+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:51.999 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:51.994+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:52.164 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:52.156+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:52.325 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:52.317+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:52.345 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:52.339+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:52.780 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:52.771+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:52.784 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:52.776+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:52.864 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:52.858+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:53.031 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:53.025+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:53.032 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:53.027+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:53.219 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:53.210+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:53.300 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:53.291+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:53.391 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:53.385+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:53.823 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:53.809+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:53.830 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:53.823+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:53.858 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:53.852+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:54.046 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:54.039+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:54.048 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:54.043+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:54.182 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:54.174+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:54.290 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:54.282+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:54.378 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:54.362+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:54.661 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:48:54.650+0000 20e22700 -1 RGWWatcher::handle_error cookie 5470989024 err (107) Transport endpoint is not connected 2019-11-29T18:48:54.823 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:54.818+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:54.828 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:54.820+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:54.871 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:54.863+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:55.099 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:55.093+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:55.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:55.096+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:55.202 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:55.194+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:55.314 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:55.306+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:55.326 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:55.320+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:55.793 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:55.785+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:55.834 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:55.827+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:55.876 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:55.871+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:56.072 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:56.065+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:56.080 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:56.068+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:56.201 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:56.194+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:56.359 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:56.350+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:56.374 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:56.369+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:56.832 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:56.824+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:56.853 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:56.844+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:56.855 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:56.849+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:57.118 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:57.112+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:57.119 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:57.114+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:57.181 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:57.172+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:57.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:57.366+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:57.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:57.424+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:57.888 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:57.875+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:57.898 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:57.890+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:57.904 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:57.900+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:58.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:58.094+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:58.101 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:58.096+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:58.178 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:58.170+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:58.349 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:58.340+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:58.402 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:58.397+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:58.880 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:58.872+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:58.921 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:58.910+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:58.947 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:58.941+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:59.093 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:59.086+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:48:59.093 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:48:59.089+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:48:59.198 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:48:59.190+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:48:59.360 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:48:59.352+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:48:59.431 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:48:59.424+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:48:59.855 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:48:59.846+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:48:59.922 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:48:59.914+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:48:59.931 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:59.924+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:48:59.932 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:48:59.927+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:00.100 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:00.094+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:00.102 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:00.097+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:00.180 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:00.172+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:00.371 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:00.363+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:00.428 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:00.422+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:00.816 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:00.808+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:00.910 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:00.902+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:00.940 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:00.935+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:00.941 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:00.936+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:01.083 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:01.077+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:01.088 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:01.083+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:01.169 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:01.162+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:01.350 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:01.342+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:01.453 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:01.446+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:01.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:01.818+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:01.920 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:01.913+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:01.949 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:01.943+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:01.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:01.945+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:02.077 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:02.070+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:02.078 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:02.073+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:02.182 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:02.175+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:02.372 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:02.365+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:02.468 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:02.463+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:02.798 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:02.792+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:02.890 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:02.883+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:03.007 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:03.001+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:03.009 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:03.004+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:03.043 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:03.037+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:03.044 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:03.039+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:03.226 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:03.219+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:03.372 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:03.364+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:03.524 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:03.517+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:03.816 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:03.808+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:03.887 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:03.880+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:04.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:04.015+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:04.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:04.017+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:04.072 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:04.067+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:04.075 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:04.068+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:04.193 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:04.185+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:04.331 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:04.324+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:04.551 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:04.545+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:04.829 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:04.820+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:04.929 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:04.921+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:04.992 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:04.987+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:04.994 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:04.990+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:05.113 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:05.107+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:05.114 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:05.109+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:05.221 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:05.213+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:05.302 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:05.294+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:05.580 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:05.574+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:05.859 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:05.852+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:05.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:05.944+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:05.950 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:05.946+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:05.965 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:05.958+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:06.168 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:06.160+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:06.169 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:06.164+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:06.258 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:06.250+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:06.261 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:06.253+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:06.582 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:06.576+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:06.910 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:06.902+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:06.957 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:06.950+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:06.960 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:06.952+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:06.986 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:06.978+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:07.222 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:07.215+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:07.223 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:07.218+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:07.262 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:07.254+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:07.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:07.300+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:07.622 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:07.616+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:07.969 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:07.946+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:08.004 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:07.994+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:08.004 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:07.997+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:08.007 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:08.000+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:08.232 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:08.226+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:08.236 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:08.230+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:08.289 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:08.282+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:08.337 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:08.329+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:08.613 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:08.605+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:08.985 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:08.978+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:09.019 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:09.012+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:09.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:09.038+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:09.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:09.041+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:09.279 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:09.270+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:09.280 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:09.274+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:09.330 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:09.323+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:09.336 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:09.329+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:09.611 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:09.604+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:09.994 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:09.984+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:10.018 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:10.011+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:10.083 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:10.077+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:10.086 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:10.081+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:10.299 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:10.292+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:10.321 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:10.313+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:10.322 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:10.315+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:10.323 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:10.316+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:10.670 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:10.656+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:10.973 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:10.965+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:11.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:11.046+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:11.057 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:11.049+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:11.072 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:11.065+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:11.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:11.300+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:11.316 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:11.309+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:11.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:11.318+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:11.327 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:11.321+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:11.685 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:11.678+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:11.970 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:11.962+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:12.085 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:12.079+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:12.085 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:12.080+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:12.108 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:12.100+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:12.306 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:12.299+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:12.309 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:12.303+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:12.310 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:12.304+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:12.325 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:12.318+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:12.638 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:49:12.630+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e513 v513) 2019-11-29T18:49:12.649 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:12.643+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:12.984 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:12.976+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:13.076 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:13.070+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:13.078 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:13.073+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:13.154 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:13.146+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:13.277 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:13.270+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:13.285 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:13.278+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:13.285 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:13.280+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:13.326 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:13.319+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:13.641 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:13.634+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:13.991 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:13.983+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:14.051 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:14.044+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:14.051 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:14.046+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:14.138 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:14.130+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:14.281 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:14.273+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:14.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:14.316+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:14.326 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:14.320+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:14.351 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:14.344+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:14.631 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:14.624+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:14.981 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:14.973+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:15.092 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:15.084+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:15.093 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:15.086+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:15.104 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:15.096+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:15.264 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:15.253+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:15.285 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:15.278+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:15.286 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:15.281+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:15.367 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:15.360+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:15.599 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:15.592+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:15.996 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:15.974+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:16.081 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:16.072+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:16.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:16.109+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:16.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:16.111+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:16.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:16.255+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:16.262 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:16.256+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:16.313 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:16.306+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:16.390 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:16.382+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:16.638 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:16.632+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:17.016 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:17.009+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:17.109 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:17.102+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:17.166 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:17.159+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:17.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:17.161+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:17.271 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:17.265+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:17.272 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:17.265+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:17.274 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:17.268+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:17.442 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:17.436+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:17.602 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:17.595+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:17.648 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:49:17.640+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e513 v513) 2019-11-29T18:49:18.007 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:17.999+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:18.096 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:18.089+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:18.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:18.205+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:18.215 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:18.209+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:18.252 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:18.244+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:18.254 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:18.247+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:18.311 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:18.304+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:18.468 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:18.460+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:18.558 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:18.551+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:18.999 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:18.991+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:19.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:19.090+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:19.182 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:19.174+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:19.183 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:19.177+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:19.298 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:19.290+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:19.304 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:19.296+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:19.305 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:19.299+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:19.471 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:19.464+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:19.516 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:19.505+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:19.960 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:19.953+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:20.147 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:20.141+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:20.160 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:20.154+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:20.163 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:20.157+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:20.276 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:20.269+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:20.279 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:20.273+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:20.333 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:20.325+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:20.503 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:20.494+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:20.513 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:20.505+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:20.919 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:20.912+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:21.113 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:21.106+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:21.169 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:21.162+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:21.172 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:21.166+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:21.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:21.283+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:21.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:21.285+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:21.315 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:21.308+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:21.472 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:21.464+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:21.515 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:21.508+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:21.925 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:21.918+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:22.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:49:22.088+0000 1694c700 -1 osd.6 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.743621+0000 front 2019-11-29T18:47:58.725677+0000 (oldest deadline 2019-11-29T18:48:40.274160+0000) 2019-11-29T18:49:22.142 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:22.135+0000 1694c700 -1 osd.3 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.070630+0000 front 2019-11-29T18:47:58.289566+0000 (oldest deadline 2019-11-29T18:48:41.444132+0000) 2019-11-29T18:49:22.145 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:22.140+0000 1694c700 -1 osd.3 513 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:22.310 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:22.303+0000 1694c700 -1 osd.0 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:48:12.208929+0000 (oldest deadline 2019-11-29T18:48:52.208929+0000) 2019-11-29T18:49:22.311 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:49:22.305+0000 1694c700 -1 osd.0 513 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e512) 2019-11-29T18:49:22.328 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:49:22.319+0000 1694c700 -1 osd.7 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.017434+0000 front 2019-11-29T18:47:58.991410+0000 (oldest deadline 2019-11-29T18:48:44.598118+0000) 2019-11-29T18:49:22.438 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:49:22.431+0000 1694c700 -1 osd.5 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:57.142514+0000 front 2019-11-29T18:47:56.984133+0000 (oldest deadline 2019-11-29T18:48:41.618552+0000) 2019-11-29T18:49:22.569 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:49:22.557+0000 1694c700 -1 osd.2 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:58.745936+0000 front 2019-11-29T18:47:58.745373+0000 (oldest deadline 2019-11-29T18:48:42.728734+0000) 2019-11-29T18:49:22.700 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:49:22.693+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 42sec e513 v513) 2019-11-29T18:49:22.912 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:49:22.907+0000 1694c700 -1 osd.4 513 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:47:59.187415+0000 front 2019-11-29T18:47:59.230300+0000 (oldest deadline 2019-11-29T18:48:40.603652+0000) 2019-11-29T18:49:23.282 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:23.272+0000 1694c700 -1 osd.3 514 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:49:24.519 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:49:24.503+0000 1694c700 -1 osd.3 515 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112587 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5470989024] snapc 0=[] ondisk+write+known_if_redirected e513) 2019-11-29T18:50:22.977 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:50:22.838+0000 1694c700 -1 osd.0 518 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112683 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e516) 2019-11-29T18:50:23.878 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:50:23.874+0000 1694c700 -1 osd.0 518 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112683 7.9 7:978e9b11:::pubsub.user.foo.client.0.bucket.%ef%84%90%e8%94%89%e8%94%8a%ce%a9%e0%a4%83%eb%8c%8b%eb%8c%8d%e5%98%88%e9%90%8b%e9%90%8b%ce%a9%e1%a0%83%e1%a0%82%e8%94%89%e3%9c%85%2f42bb32be-1e05-491f-815b-a606bdf24d77.4355.612:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e516) 2019-11-29T18:51:15.117 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:15.105+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:15.282 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:15.273+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:16.089 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:16.081+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:16.174 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:16.168+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:16.293 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:16.285+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:16.569 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:16.564+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:17.067 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:17.059+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:17.133 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:17.126+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:17.293 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:17.283+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:17.570 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:17.564+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:17.985 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:17.977+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:18.051 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:18.043+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:18.171 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:18.166+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:18.314 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:18.308+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:18.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:18.527+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:18.734 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:18.724+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:18.948 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:18.940+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:19.020 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:19.002+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:19.221 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:19.211+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:19.338 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:19.332+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:19.546 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:19.539+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:19.753 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:19.746+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:19.990 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:19.983+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:20.007 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:20.000+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:20.229 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:20.223+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:20.323 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:20.316+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:20.576 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:20.569+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:20.783 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:20.776+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:20.999 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:20.993+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:21.030 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:21.024+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:21.255 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:21.250+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:21.311 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:21.303+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:21.561 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:21.552+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:21.763 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:21.756+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:22.026 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:22.018+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:22.054 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:22.045+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:22.213 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:22.205+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:22.334 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:22.326+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:22.580 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:22.574+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:22.777 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:22.770+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:23.013 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:23.004+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:23.061 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:23.054+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:23.243 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:23.228+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:23.300 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:23.293+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:23.605 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:23.598+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:23.821 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:23.814+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:24.041 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:24.033+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:24.114 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:24.107+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:24.266 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:24.259+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:24.344 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:24.336+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:24.577 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:24.568+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:24.808 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:24.799+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:25.080 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:25.065+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:25.107 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:25.091+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:25.236 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:25.228+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:25.338 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:25.329+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:25.625 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:25.618+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:25.833 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:25.825+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:26.099 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:26.090+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:26.139 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:26.130+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:26.252 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:26.243+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:26.347 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:26.339+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:26.650 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:26.644+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:26.871 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:26.863+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:27.097 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:27.089+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:27.103 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:27.094+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:27.278 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:27.271+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:27.361 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:27.352+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:27.606 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:27.599+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:27.882 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:27.873+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:28.136 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:28.127+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:28.156 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:28.147+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:28.293 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:28.286+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:28.383 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:28.373+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:28.631 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:28.623+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:28.867 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:28.858+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:29.116 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:29.108+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:29.133 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:29.125+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:29.253 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:29.246+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:29.397 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:29.389+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:29.599 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:29.586+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:29.847 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:29.839+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:30.095 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:30.087+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:30.109 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:30.101+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:30.257 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:30.250+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:30.379 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:30.371+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:30.640 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:30.627+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:30.860 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:30.851+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:31.078 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:31.070+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:31.135 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:31.120+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:31.221 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:31.214+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:31.408 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:38:44.297 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.415 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:38:44.302 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.418 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:31.410+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:31.428 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:38:44.314 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.431 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:38:44.319 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.438 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:38:44.211 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.451 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:38:44.221 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.612 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:38:44.499 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.615 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:38:44.503 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.620 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:31.613+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:31.719 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:38:44.492 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:51:31.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:31.834+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:32.103 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:32.095+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:32.166 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:32.158+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:32.185 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:32.179+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:32.416 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:32.408+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:32.589 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:32.582+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:32.850 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:32.842+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:33.148 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:33.133+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:33.195 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:33.188+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:33.198 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:33.190+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:33.411 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:33.402+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:33.568 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:33.561+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:33.898 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:33.890+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:34.133 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:34.126+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:34.233 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:34.227+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:34.237 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:34.230+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:34.444 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:34.436+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:34.596 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:34.588+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:34.954 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:34.946+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:35.190 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:35.181+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:35.216 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:35.209+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:35.243 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:35.235+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:35.410 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:35.401+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:35.615 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:35.608+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:35.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:35.953+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:36.175 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:36.168+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:36.239 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:36.233+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:36.253 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:36.247+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:36.446 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:36.439+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:36.650 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:36.642+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:36.971 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:36.963+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:37.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:37.143+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:37.195 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:37.187+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:37.278 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:37.270+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:37.457 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:37.446+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:37.652 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:37.645+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:38.004 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:37.994+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:38.166 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:38.158+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:38.209 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:38.184+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:38.262 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:38.254+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:38.468 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:38.460+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:38.632 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:38.625+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:38.993 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:38.985+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:39.159 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:39.151+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:39.181 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:39.173+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:39.288 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:39.277+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:39.487 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:39.479+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:39.634 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:39.626+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:39.976 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:39.968+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:40.129 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:40.122+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:40.221 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:40.214+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:40.319 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:40.311+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:40.529 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:40.521+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:40.604 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:40.597+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:40.966 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:40.959+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:41.096 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:41.088+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:41.196 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:41.188+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:41.290 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:41.283+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:41.550 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:41.542+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:41.568 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:41.561+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:42.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:42.019+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:42.100 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:42.093+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:42.248 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:42.240+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:42.277 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:42.267+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:42.558 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:42.550+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:42.616 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:42.609+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:43.013 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:43.006+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:43.149 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:43.143+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:43.217 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:43.210+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:43.329 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:43.321+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:43.534 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:43.526+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:43.580 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:43.565+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:44.037 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:44.026+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:44.114 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:44.108+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:44.187 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:44.180+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:44.373 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:44.363+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:44.507 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:44.499+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:44.588 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:44.581+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:45.046 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:45.038+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:45.123 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:45.118+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:45.176 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:45.168+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:45.402 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:45.395+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:45.493 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:45.483+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:45.622 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:45.617+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:46.011 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:46.004+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:46.152 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:46.147+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:46.197 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:46.189+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:46.411 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:46.401+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:46.473 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:46.466+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:46.617 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:46.611+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:46.981 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:46.973+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:47.125 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:47.117+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:47.193 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:47.186+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:47.379 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:47.372+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:47.460 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:47.453+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:51:47.654 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:51:47.648+0000 1694c700 -1 osd.3 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.815284+0000 front 2019-11-29T18:50:31.831098+0000 (oldest deadline 2019-11-29T18:51:16.470419+0000) 2019-11-29T18:51:48.014 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:51:48.008+0000 1694c700 -1 osd.5 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.104035+0000 front 2019-11-29T18:50:32.227711+0000 (oldest deadline 2019-11-29T18:51:18.120567+0000) 2019-11-29T18:51:48.165 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:51:48.159+0000 1694c700 -1 osd.2 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:34.616718+0000 front 2019-11-29T18:50:34.619650+0000 (oldest deadline 2019-11-29T18:51:15.942065+0000) 2019-11-29T18:51:48.168 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:51:48.156+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.0 [v2:172.21.15.69:6810/9729,v1:172.21.15.69:6811/9729] for 46sec e518 v518) 2019-11-29T18:51:48.183 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:51:48.178+0000 1694c700 -1 osd.1 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:31.480487+0000 front 2019-11-29T18:50:31.777775+0000 (oldest deadline 2019-11-29T18:51:48.053175+0000) 2019-11-29T18:51:48.256 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:51:48.247+0000 1694c700 -1 osd.6 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:33.361355+0000 front 2019-11-29T18:50:34.028284+0000 (oldest deadline 2019-11-29T18:51:17.454016+0000) 2019-11-29T18:51:48.415 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:51:48.408+0000 1694c700 -1 osd.7 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.875436+0000 front 2019-11-29T18:50:29.608644+0000 (oldest deadline 2019-11-29T18:51:14.883114+0000) 2019-11-29T18:51:48.462 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:51:48.456+0000 1694c700 -1 osd.4 518 heartbeat_check: no reply from 172.21.15.69:6814 osd.0 since back 2019-11-29T18:50:29.414961+0000 front 2019-11-29T18:50:29.417360+0000 (oldest deadline 2019-11-29T18:51:15.272439+0000) 2019-11-29T18:52:21.028 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:21.023+0000 35273700 -1 osd.5 522 build_incremental_map_msg missing incremental map 522 2019-11-29T18:52:21.042 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:21.027+0000 38a7a700 -1 osd.5 522 build_incremental_map_msg missing incremental map 522 2019-11-29T18:52:21.315 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:21.221+0000 39a7c700 -1 osd.5 522 build_incremental_map_msg missing incremental map 522 2019-11-29T18:52:21.436 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:21.421+0000 3b27f700 -1 osd.5 522 build_incremental_map_msg missing incremental map 522 2019-11-29T18:52:46.128 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:46.120+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:46.307 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:46.299+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:46.732 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:46.723+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:47.157 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:47.149+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:47.322 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:47.313+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:47.776 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:47.766+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:48.164 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:48.155+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:48.345 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:48.327+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:48.801 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:48.792+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:49.178 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:49.170+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:49.324 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:49.316+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:49.770 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:49.762+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:50.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:50.004+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:50.138 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:50.129+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:50.219 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:50.213+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:50.223 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:50.215+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:50.308 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:50.300+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:50.823 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:50.816+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:51.008 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:51.001+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:51.145 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:51.137+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:51.212 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:51.207+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:51.215 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:51.208+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:51.357 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:51.350+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:51.553 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:52:51.488+0000 20e22700 -1 RGWWatcher::handle_error cookie 5456422688 err (107) Transport endpoint is not connected 2019-11-29T18:52:51.868 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:51.861+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:52.026 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:52.020+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:52.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:52.181+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:52.192 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:52.185+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:52.215 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:52.206+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:52.321 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:52.317+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:52.381 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:52.374+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:52.824 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:52.816+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:53.002 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:52.997+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:53.166 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:53.160+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:53.229 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:53.222+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:53.265 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:53.257+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:53.347 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:53.342+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:53.386 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:53.372+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:53.831 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:53.825+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:54.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:54.040+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:54.179 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:54.173+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:54.259 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:54.251+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:54.271 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:54.263+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:54.320 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:54.315+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:54.393 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:54.384+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:54.813 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:54.807+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:55.003 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:54.997+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:55.215 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:55.208+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:55.221 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:55.208+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:55.298 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:55.290+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:55.333 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:55.323+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:55.442 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:55.432+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:55.842 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:55.833+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:56.012 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:56.004+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:56.183 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:56.173+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:56.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:56.180+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:56.303 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:56.294+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:56.330 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:56.323+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:56.495 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:56.489+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:56.893 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:56.884+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:57.033 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:57.025+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:57.147 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:57.139+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:57.162 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:57.156+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:57.276 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:57.271+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:57.345 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:57.339+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:57.535 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:57.527+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:57.899 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:57.892+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:58.046 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:58.039+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:58.112 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:58.104+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:58.207 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:58.200+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:58.244 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:58.238+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:58.331 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:58.322+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:58.540 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:58.529+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:58.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:58.881+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:52:59.071 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:52:59.063+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:52:59.161 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:52:59.151+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:52:59.184 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:52:59.177+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:52:59.269 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:59.262+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:52:59.270 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:52:59.264+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:52:59.343 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:52:59.335+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:52:59.515 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:52:59.509+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:52:59.917 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:52:59.910+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:00.056 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:00.045+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:00.154 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:00.146+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:00.213 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:00.205+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:00.240 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:00.231+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:00.250 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:00.233+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:00.362 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:00.353+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:00.560 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:00.553+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:00.929 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:00.922+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:01.040 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:01.016+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:01.182 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:01.174+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:01.192 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:01.184+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:01.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:01.187+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:01.229 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:01.221+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:01.332 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:01.325+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:01.569 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:01.562+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:01.899 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:01.892+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:02.013 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:02.004+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:02.182 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:02.175+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:02.197 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:02.188+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:02.198 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:02.192+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:02.216 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:02.209+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:02.373 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:02.364+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:02.526 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:02.518+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:02.860 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:02.852+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:03.047 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:03.040+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:03.178 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:03.170+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:03.178 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:03.171+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:03.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:03.219+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:03.232 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:03.222+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:03.371 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:03.362+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:03.465 INFO:tasks.rgw.client.0.smithi069.stdout:2019-11-29T18:53:03.368+0000 20e22700 -1 RGWWatcher::handle_error cookie 5414036224 err (107) Transport endpoint is not connected 2019-11-29T18:53:03.530 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:03.523+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:03.827 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:03.819+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:04.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:04.015+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:04.146 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:04.139+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:04.146 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:04.138+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:04.217 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:04.208+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:04.218 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:04.212+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:04.393 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:04.383+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:04.579 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:04.571+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:04.875 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:04.868+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:05.021 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:05.014+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:05.157 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:05.151+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:05.169 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:05.160+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:05.260 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:05.253+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:05.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:05.256+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:05.375 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:05.366+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:05.617 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:05.607+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:05.887 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:05.873+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:06.065 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:06.059+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:06.197 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:06.191+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:06.204 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:06.196+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:06.224 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:06.217+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:06.226 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:06.221+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:06.409 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:06.402+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:06.618 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:06.612+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:06.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:06.902+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:07.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:07.087+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:07.097 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:07.090+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:07.191 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:07.182+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:07.194 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:07.186+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:07.234 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:07.227+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:07.249 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:07.242+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:07.461 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:07.452+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:07.649 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:07.641+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:07.888 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:07.875+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:08.063 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:08.057+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:08.064 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:08.058+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:08.231 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:08.223+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:08.232 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:08.225+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:08.240 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:08.233+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:08.265 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:08.258+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:08.425 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:08.416+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:08.670 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:08.659+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:08.929 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:08.922+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:09.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:09.053+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:09.060 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:09.055+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:09.227 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:09.221+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:09.228 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:09.223+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:09.241 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:09.229+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:09.311 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:09.303+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:09.400 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:09.392+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:09.639 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:09.618+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:09.911 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:09.904+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:10.078 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:10.071+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:10.079 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:10.073+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:10.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:10.191+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:10.263 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:10.257+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:10.265 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:10.260+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:10.272 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:10.266+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:10.356 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:10.348+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:10.661 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:10.648+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:10.954 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:10.947+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:11.102 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:11.093+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:11.102 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:11.095+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:11.241 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:11.234+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:11.292 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:11.286+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:11.300 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:11.288+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:11.307 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:11.300+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:11.359 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:11.352+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:11.663 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:11.656+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:11.969 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:11.962+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:12.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:12.093+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:12.101 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:12.095+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:12.204 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:12.197+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:12.284 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:12.276+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:12.324 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:12.316+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:12.325 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:12.319+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:12.326 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:12.319+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:12.647 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:12.640+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:12.950 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:12.943+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:13.144 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:13.137+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:13.145 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:13.139+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:13.183 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:13.176+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:13.260 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:13.253+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:13.291 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:13.284+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:13.317 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:13.310+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:13.318 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:13.313+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:13.632 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:13.625+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:13.954 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:13.947+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:14.158 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:14.151+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:14.158 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:14.153+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:14.198 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:14.191+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:14.266 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:14.260+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:14.290 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:14.285+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:14.294 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:14.289+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:14.337 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:14.325+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:14.626 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:14.618+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:14.975 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:14.969+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:15.190 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:15.184+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:15.191 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:15.186+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:15.221 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:15.204+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:15.284 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:15.278+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:15.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:15.291+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:15.299 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:15.295+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:15.389 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:15.382+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:15.655 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:15.647+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:15.961 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:15.953+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:16.173 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:16.163+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:16.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:16.181+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:16.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:16.184+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:16.295 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:16.289+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:16.309 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:16.303+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:16.310 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:16.305+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:16.383 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:16.376+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:16.691 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:16.684+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:16.951 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:16.943+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:17.129 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:17.122+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:17.143 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:17.138+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:17.146 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:17.141+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:17.268 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:17.263+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:17.277 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:17.265+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:17.314 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:17.310+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:17.371 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:17.364+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:17.691 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:17.684+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:18.002 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:17.995+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:18.113 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:18.104+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:18.136 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:18.130+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:18.137 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:18.132+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:18.278 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:18.272+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:18.281 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:18.276+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:18.328 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:18.322+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:18.357 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:18.350+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:18.492 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:53:18.487+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e523 v523) 2019-11-29T18:53:18.679 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:18.672+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:18.972 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:18.964+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:19.117 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:19.112+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:19.118 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:19.113+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:19.151 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:19.138+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:19.285 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:19.280+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:19.286 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:19.281+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:19.358 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:19.352+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:19.389 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:19.381+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:19.716 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:19.709+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:19.972 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:19.963+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:20.117 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:20.108+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:20.126 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:20.122+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:20.129 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:20.124+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:20.247 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:20.239+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:20.249 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:20.242+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:20.360 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:20.354+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:20.376 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:20.368+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:20.711 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:20.701+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:21.026 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:21.013+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:21.131 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:21.122+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:21.177 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:21.172+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:21.179 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:21.175+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:21.228 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:21.223+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:21.229 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:21.224+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:21.347 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:21.337+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:21.369 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:21.364+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:21.769 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:21.759+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:22.066 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:22.058+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:22.164 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:22.159+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:22.167 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:22.162+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:22.185 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:22.177+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:22.220 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:22.213+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:22.220 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:22.216+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:22.319 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:22.311+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:22.379 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:22.372+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:22.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:22.748+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:23.099 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:23.091+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:23.188 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:23.183+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:23.192 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:23.187+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:23.197 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:23.189+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:23.217 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:23.210+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:23.218 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:23.213+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:23.344 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:23.335+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:23.344 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:23.334+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:23.499 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:53:23.495+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e523 v523) 2019-11-29T18:53:23.732 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:23.720+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:24.135 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:24.125+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:24.164 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:24.156+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:24.189 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:24.181+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:24.190 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:24.185+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:24.211 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:24.205+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:24.215 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:24.210+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:24.314 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:24.305+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:24.332 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:24.326+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:24.710 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:24.702+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:25.139 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:25.130+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:25.151 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:25.143+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:25.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:25.189+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:25.195 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:25.191+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:25.259 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:25.251+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:25.261 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:25.255+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:25.300 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:25.294+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:25.323 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:25.315+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:25.696 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:25.687+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:26.149 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:26.140+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:26.203 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:26.195+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:26.244 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:26.214+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:26.245 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:26.217+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:26.286 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:26.280+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:26.287 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:26.282+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:26.301 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:26.293+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:26.309 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:26.303+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:26.725 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:26.717+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:27.158 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:27.151+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:27.195 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:27.186+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:27.210 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:27.204+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:27.212 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:27.206+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:27.301 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:27.294+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:27.303 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:27.298+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:27.357 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:27.349+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:27.362 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:27.357+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:27.757 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:27.749+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:28.171 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:28.166+0000 1694c700 -1 osd.3 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.650136+0000 front 2019-11-29T18:52:04.997784+0000 (oldest deadline 2019-11-29T18:52:49.329899+0000) 2019-11-29T18:53:28.172 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:28.167+0000 1694c700 -1 osd.3 523 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:28.188 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T18:53:28.181+0000 1694c700 -1 osd.4 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.895121+0000 front 2019-11-29T18:52:05.932292+0000 (oldest deadline 2019-11-29T18:52:49.168487+0000) 2019-11-29T18:53:28.219 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T18:53:28.211+0000 1694c700 -1 osd.5 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:04.652351+0000 front 2019-11-29T18:52:04.994281+0000 (oldest deadline 2019-11-29T18:52:46.169667+0000) 2019-11-29T18:53:28.343 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:28.337+0000 1694c700 -1 osd.0 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 ever on either front or back, first ping sent 2019-11-29T18:52:18.872607+0000 (oldest deadline 2019-11-29T18:52:58.872607+0000) 2019-11-29T18:53:28.348 INFO:tasks.ceph.osd.0.smithi069.stderr:2019-11-29T18:53:28.342+0000 1694c700 -1 osd.0 523 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:80490 7.9 7.31bb659 (undecoded) ondisk+retry+read+known_if_redirected e522) 2019-11-29T18:53:28.386 INFO:tasks.ceph.osd.6.smithi163.stderr:2019-11-29T18:53:28.378+0000 1694c700 -1 osd.6 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:01.434177+0000 front 2019-11-29T18:52:00.759825+0000 (oldest deadline 2019-11-29T18:52:46.029447+0000) 2019-11-29T18:53:28.425 INFO:tasks.ceph.osd.2.smithi069.stderr:2019-11-29T18:53:28.410+0000 1694c700 -1 osd.2 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:05.338221+0000 front 2019-11-29T18:52:05.363930+0000 (oldest deadline 2019-11-29T18:52:49.966770+0000) 2019-11-29T18:53:28.550 INFO:tasks.ceph.mon.a.smithi069.stderr:2019-11-29T18:53:28.542+0000 1a899700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.69:6802/9723,v1:172.21.15.69:6803/9723] for 46sec e523 v523) 2019-11-29T18:53:28.731 INFO:tasks.ceph.osd.7.smithi163.stderr:2019-11-29T18:53:28.725+0000 1694c700 -1 osd.7 523 heartbeat_check: no reply from 172.21.15.69:6806 osd.1 since back 2019-11-29T18:52:00.574951+0000 front 2019-11-29T18:52:00.382219+0000 (oldest deadline 2019-11-29T18:52:46.238951+0000) 2019-11-29T18:53:29.260 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:29.202+0000 1694c700 -1 osd.3 524 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:30.517 INFO:tasks.ceph.osd.3.smithi069.stderr:2019-11-29T18:53:30.402+0000 1694c700 -1 osd.3 525 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4465.0:112974 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 5414036224] snapc 0=[] ondisk+write+known_if_redirected e523) 2019-11-29T18:53:56.203 INFO:tasks.ceph.osd.1.smithi069.stderr:*** Caught signal (Segmentation fault) ** 2019-11-29T18:53:56.203 INFO:tasks.ceph.osd.1.smithi069.stderr: in thread 114ab700 thread_name:msgr-worker-0 2019-11-29T18:53:56.288 INFO:tasks.ceph.osd.1.smithi069.stderr: ceph version 15.0.0-7809-g656c8e8 (656c8e8049c2c1acd363143c842c2edf1fe09b64) octopus (dev) 2019-11-29T18:53:56.288 INFO:tasks.ceph.osd.1.smithi069.stderr: 1: (()+0xf5f0) [0xc09b5f0] 2019-11-29T18:53:56.288 INFO:tasks.ceph.osd.1.smithi069.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe89561] 2019-11-29T18:53:56.288 INFO:tasks.ceph.osd.1.smithi069.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100abc0] 2019-11-29T18:53:56.294 INFO:tasks.ceph.osd.1.smithi069.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1017963] 2019-11-29T18:53:56.294 INFO:tasks.ceph.osd.1.smithi069.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x1017be0] 2019-11-29T18:53:56.304 INFO:tasks.ceph.osd.1.smithi069.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101813d] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1000dc4] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 8: (AsyncConnection::process()+0x794) [0xfdb414] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3c905] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 10: (()+0xd3a317) [0xe42317] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 11: (()+0x12121ff) [0x131a1ff] 2019-11-29T18:53:56.315 INFO:tasks.ceph.osd.1.smithi069.stderr: 12: (()+0x7e65) [0xc093e65] 2019-11-29T18:53:56.316 INFO:tasks.ceph.osd.1.smithi069.stderr: 13: (clone()+0x6d) [0xd20b88d] 2019-11-29T18:53:56.322 INFO:tasks.ceph.osd.1.smithi069.stderr:2019-11-29T18:53:56.314+0000 114ab700 -1 *** Caught signal (Segmentation fault) ** 2019-11-29T18:53:56.322 INFO:tasks.ceph.osd.1.smithi069.stderr: in thread 114ab700 thread_name:msgr-worker-0 2019-11-29T18:53:56.322 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:53:56.322 INFO:tasks.ceph.osd.1.smithi069.stderr: ceph version 15.0.0-7809-g656c8e8 (656c8e8049c2c1acd363143c842c2edf1fe09b64) octopus (dev) 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 1: (()+0xf5f0) [0xc09b5f0] 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe89561] 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100abc0] 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1017963] 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x1017be0] 2019-11-29T18:53:56.323 INFO:tasks.ceph.osd.1.smithi069.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101813d] 2019-11-29T18:53:56.324 INFO:tasks.ceph.osd.1.smithi069.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1000dc4] 2019-11-29T18:53:56.324 INFO:tasks.ceph.osd.1.smithi069.stderr: 8: (AsyncConnection::process()+0x794) [0xfdb414] 2019-11-29T18:53:56.324 INFO:tasks.ceph.osd.1.smithi069.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3c905] 2019-11-29T18:53:56.324 INFO:tasks.ceph.osd.1.smithi069.stderr: 10: (()+0xd3a317) [0xe42317] 2019-11-29T18:53:56.324 INFO:tasks.ceph.osd.1.smithi069.stderr: 11: (()+0x12121ff) [0x131a1ff] 2019-11-29T18:53:56.325 INFO:tasks.ceph.osd.1.smithi069.stderr: 12: (()+0x7e65) [0xc093e65] 2019-11-29T18:53:56.325 INFO:tasks.ceph.osd.1.smithi069.stderr: 13: (clone()+0x6d) [0xd20b88d] 2019-11-29T18:53:56.325 INFO:tasks.ceph.osd.1.smithi069.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-11-29T18:53:56.325 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:53:56.725 INFO:tasks.ceph.osd.1.smithi069.stderr: 0> 2019-11-29T18:53:56.314+0000 114ab700 -1 *** Caught signal (Segmentation fault) ** 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: in thread 114ab700 thread_name:msgr-worker-0 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: ceph version 15.0.0-7809-g656c8e8 (656c8e8049c2c1acd363143c842c2edf1fe09b64) octopus (dev) 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: 1: (()+0xf5f0) [0xc09b5f0] 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe89561] 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100abc0] 2019-11-29T18:53:56.726 INFO:tasks.ceph.osd.1.smithi069.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1017963] 2019-11-29T18:53:56.727 INFO:tasks.ceph.osd.1.smithi069.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x1017be0] 2019-11-29T18:53:56.727 INFO:tasks.ceph.osd.1.smithi069.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101813d] 2019-11-29T18:53:56.727 INFO:tasks.ceph.osd.1.smithi069.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1000dc4] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 8: (AsyncConnection::process()+0x794) [0xfdb414] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3c905] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 10: (()+0xd3a317) [0xe42317] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 11: (()+0x12121ff) [0x131a1ff] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 12: (()+0x7e65) [0xc093e65] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: 13: (clone()+0x6d) [0xd20b88d] 2019-11-29T18:53:56.728 INFO:tasks.ceph.osd.1.smithi069.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-11-29T18:53:56.729 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:53:57.193 INFO:tasks.ceph.osd.1.smithi069.stderr: -742> 2019-11-29T18:53:56.314+0000 114ab700 -1 *** Caught signal (Segmentation fault) ** 2019-11-29T18:53:57.193 INFO:tasks.ceph.osd.1.smithi069.stderr: in thread 114ab700 thread_name:msgr-worker-0 2019-11-29T18:53:57.193 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:53:57.194 INFO:tasks.ceph.osd.1.smithi069.stderr: ceph version 15.0.0-7809-g656c8e8 (656c8e8049c2c1acd363143c842c2edf1fe09b64) octopus (dev) 2019-11-29T18:53:57.194 INFO:tasks.ceph.osd.1.smithi069.stderr: 1: (()+0xf5f0) [0xc09b5f0] 2019-11-29T18:53:57.194 INFO:tasks.ceph.osd.1.smithi069.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe89561] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100abc0] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1017963] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x1017be0] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101813d] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1000dc4] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 8: (AsyncConnection::process()+0x794) [0xfdb414] 2019-11-29T18:53:57.195 INFO:tasks.ceph.osd.1.smithi069.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3c905] 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: 10: (()+0xd3a317) [0xe42317] 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: 11: (()+0x12121ff) [0x131a1ff] 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: 12: (()+0x7e65) [0xc093e65] 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: 13: (clone()+0x6d) [0xd20b88d] 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-11-29T18:53:57.196 INFO:tasks.ceph.osd.1.smithi069.stderr: 2019-11-29T18:54:00.200 INFO:tasks.ceph.osd.1.smithi069.stderr:daemon-helper: command crashed with signal 11 2019-11-29T18:54:02.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2019-11-29T18:54:08.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2019-11-29T18:54:08.880 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:41:21.767 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:08.919 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:41:21.808 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:08.929 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:41:21.816 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:08.936 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:41:21.820 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:08.962 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:41:21.734 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:08.966 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:41:21.739 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:09.078 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:41:21.961 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:09.115 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:41:22.004 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:09.201 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:41:21.974 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:54:14.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~13s 2019-11-29T18:54:21.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~19s 2019-11-29T18:54:27.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2019-11-29T18:54:33.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~32s 2019-11-29T18:54:40.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~38s 2019-11-29T18:54:46.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~44s 2019-11-29T18:54:52.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~51s 2019-11-29T18:54:59.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~57s 2019-11-29T18:55:05.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~63s 2019-11-29T18:55:11.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~69s 2019-11-29T18:55:18.117 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~76s 2019-11-29T18:55:24.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~82s 2019-11-29T18:55:30.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~88s 2019-11-29T18:55:34.227 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:42:47.116 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.233 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:42:47.120 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.248 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:42:47.133 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.252 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:42:47.138 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.255 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:42:47.028 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.264 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:42:47.037 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.309 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:42:47.198 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.383 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:42:47.270 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:34.435 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:42:47.208 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T18:55:37.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2019-11-29T18:55:43.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~101s 2019-11-29T18:55:49.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~107s 2019-11-29T18:55:56.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~114s 2019-11-29T18:56:02.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~120s 2019-11-29T18:56:08.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~126s 2019-11-29T18:56:14.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~133s 2019-11-29T18:56:21.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~139s 2019-11-29T18:56:27.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~145s 2019-11-29T18:56:33.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~152s 2019-11-29T18:56:40.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~158s 2019-11-29T18:56:46.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~164s 2019-11-29T18:56:52.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~171s 2019-11-29T18:56:59.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~177s 2019-11-29T18:57:05.501 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~183s 2019-11-29T18:57:11.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~190s 2019-11-29T18:57:18.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~196s 2019-11-29T18:57:24.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~202s 2019-11-29T18:57:30.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~208s 2019-11-29T18:57:37.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2019-11-29T18:57:43.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~221s 2019-11-29T18:57:49.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~227s 2019-11-29T18:57:56.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~234s 2019-11-29T18:58:02.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~240s 2019-11-29T18:58:08.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~246s 2019-11-29T18:58:14.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~253s 2019-11-29T18:58:21.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~259s 2019-11-29T18:58:27.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2019-11-29T18:58:33.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~272s 2019-11-29T18:58:40.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~278s 2019-11-29T18:58:46.544 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~284s 2019-11-29T18:58:52.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~291s 2019-11-29T18:58:59.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~297s 2019-11-29T18:59:05.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~303s 2019-11-29T18:59:05.484 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-11-29T18:59:05.484 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-master/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-11-29T19:00:42.506 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:47:55.394 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.516 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:47:55.405 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.523 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:47:55.411 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.525 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:47:55.413 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.529 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:47:55.302 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.537 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:47:55.310 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.733 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:47:55.621 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.767 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:47:55.656 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:00:42.830 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:47:55.603 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.829 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:49:43.718 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.833 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:49:43.722 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.883 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:49:43.771 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.885 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:49:43.773 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.906 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:49:43.795 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.907 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:49:43.680 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.909 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:49:43.682 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:30.947 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:49:43.835 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:02:31.008 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:49:43.781 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:04:37.091 INFO:tasks.ceph.osd.4.smithi163.stderr:2019-11-29T19:04:37.084+0000 36676700 -1 osd.4 536 build_incremental_map_msg missing incremental map 502 2019-11-29T19:04:42.147 INFO:tasks.ceph.osd.5.smithi163.stderr:2019-11-29T19:04:42.132+0000 34271700 -1 osd.5 539 build_incremental_map_msg missing incremental map 522 2019-11-29T19:05:01.848 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:52:14.737 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:01.853 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:52:14.741 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:01.859 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:52:14.747 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:01.862 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:52:14.750 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:01.864 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:52:14.637 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:01.869 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:52:14.642 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:02.046 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:52:14.817 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:02.057 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:52:14.946 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:05:02.114 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:52:15.001 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.204 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:55:30.093 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.321 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:55:30.065 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.321 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:55:30.092 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.321 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:55:30.119 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.396 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:55:30.284 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.399 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:55:30.287 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.422 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:55:30.310 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.545 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:55:30.318 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:08:17.621 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:55:30.509 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.113 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:56:40.001 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.123 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:56:40.012 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.146 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:56:40.034 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.150 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:56:39.923 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.153 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:56:40.041 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.156 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:56:39.929 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.182 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:56:40.071 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.231 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:56:40.119 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:09:27.246 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:56:40.019 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:52.917 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:58:05.805 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:52.954 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:58:05.842 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:52.955 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:58:05.844 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:52.955 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:58:05.843 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:53.001 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:58:05.773 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:53.013 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:58:05.787 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:53.022 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:07:58:05.910 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:53.065 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:07:58:05.954 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:10:53.099 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:07:58:05.872 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.375 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:01:50.264 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.381 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:01:50.270 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.391 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:01:50.279 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.395 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:01:50.283 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.406 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:01:50.179 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.413 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:01:50.187 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.543 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:01:50.430 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.544 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:01:50.433 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:14:37.634 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:01:50.407 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.693 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:06:27.581 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.703 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:06:27.591 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.752 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:06:27.639 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.761 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:06:27.649 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.775 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:06:27.548 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.787 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:06:27.560 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.861 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:06:27.749 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:14.880 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:06:27.766 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:15.020 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:06:27.793 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.653 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:07:00.541 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.663 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:07:00.552 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.679 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:07:00.567 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.686 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:07:00.572 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.691 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:07:00.463 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.708 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:07:00.481 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.736 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:07:00.623 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.755 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:07:00.642 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:19:47.817 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:07:00.590 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.796 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:12:10.684 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.824 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:12:10.698 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.825 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:12:10.704 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.825 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:12:10.699 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.829 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:12:10.602 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:57.835 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:12:10.608 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:58.015 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:12:10.903 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:58.047 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:12:10.934 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:24:58.113 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:12:10.886 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.338 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:14:49.226 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.360 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:14:49.249 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.379 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:14:49.267 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.383 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:14:49.270 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.407 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:14:49.179 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.432 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:14:49.320 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.446 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:14:49.219 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.470 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:14:49.357 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:27:36.631 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:14:49.404 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.046 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:17:20.934 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.057 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:17:20.946 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.078 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:17:20.851 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.079 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:17:20.967 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.085 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:17:20.858 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.087 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:17:20.974 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.207 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:17:21.096 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.216 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:17:21.103 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:30:08.325 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:17:21.098 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.210 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:22:35.099 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.236 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:22:35.103 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.236 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:22:35.112 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.236 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:22:35.117 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.250 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:22:35.023 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.263 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:22:35.036 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.417 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:22:35.302 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.445 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:22:35.333 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:22.539 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:22:35.312 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.072 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:23:10.955 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.099 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:23:10.987 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.140 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:23:10.912 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.145 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:23:11.033 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.147 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:23:10.920 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.149 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:23:11.037 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.153 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:23:11.041 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.181 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:23:11.065 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:35:58.223 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:23:10.995 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.664 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:27:45.553 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.688 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:27:45.568 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.688 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:27:45.573 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.690 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:27:45.578 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.700 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:27:45.473 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.717 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:27:45.482 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.861 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:27:45.749 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:32.950 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:27:45.838 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:40:33.057 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:27:45.830 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.739 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:31:32.627 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.791 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:31:32.678 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.803 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:31:32.575 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.804 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:31:32.691 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.805 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:31:32.578 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.805 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:31:32.693 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.908 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:31:32.795 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.919 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:31:32.691 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:44:19.956 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:31:32.842 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:43.949 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:32:56.837 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:43.957 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:32:56.846 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:43.966 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:32:56.853 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:43.968 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:32:56.856 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:44.000 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:32:56.773 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:44.009 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:32:56.782 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:44.033 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:32:56.917 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:44.038 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:32:56.925 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:45:44.183 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:32:56.952 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:55.972 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:36:08.861 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:55.993 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:36:08.870 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.049 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:36:08.936 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.051 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:36:08.939 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.108 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:36:08.881 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.112 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:36:08.885 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.159 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:36:09.047 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.179 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:36:09.066 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:48:56.272 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:36:09.034 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.183 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:38:07.071 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.196 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:38:07.085 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.204 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:38:07.092 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.210 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:38:07.098 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.212 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:38:06.985 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.232 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:38:07.006 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.286 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:38:07.175 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.298 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:38:07.186 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:50:54.376 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:38:07.149 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.689 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:39:54.577 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.709 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:39:54.597 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.734 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:39:54.622 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.744 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:39:54.632 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.785 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:39:54.554 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.785 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:39:54.660 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.788 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:39:54.561 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.801 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:39:54.684 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:52:41.886 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:39:54.659 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.678 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:43:16.567 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.684 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:43:16.573 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.693 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:43:16.581 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.701 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:43:16.588 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.707 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:43:16.481 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.713 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:43:16.486 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.834 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:43:16.721 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.880 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:43:16.768 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T19:56:03.932 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:43:16.705 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.468 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:21.357 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.474 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:21.363 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.541 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:21.427 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.543 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:21.316 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.544 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:21.430 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.552 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:21.324 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.618 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:21.506 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.688 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:21.576 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:08.712 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:21.486 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.521 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:28.404 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.527 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:28.416 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.538 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:28.426 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.542 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:28.430 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.555 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:28.328 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.556 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:48:28.444 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.569 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:28.342 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.578 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:48:28.466 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:01:15.630 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:48:28.403 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.729 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:53:44.617 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.743 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:53:44.631 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.749 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:53:44.637 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.752 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:53:44.640 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.752 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:53:44.526 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.757 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:53:44.530 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.933 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:53:44.822 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:31.993 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:53:44.881 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:06:32.063 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:53:44.836 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.185 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:56:43.074 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.226 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:56:43.115 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.241 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:56:43.125 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.251 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:56:43.139 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.252 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:56:43.025 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.263 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:56:43.032 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.323 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:56:43.211 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.363 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:56:43.136 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:09:30.384 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:56:43.272 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.611 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:58:56.500 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.616 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:58:56.505 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.633 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:58:56.521 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.638 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:58:56.526 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.639 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:58:56.413 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.665 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:58:56.438 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.707 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:08:58:56.595 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.776 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:08:58:56.664 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:11:43.810 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:08:58:56.583 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.794 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:04:06.683 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.799 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:04:06.688 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.810 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:04:06.698 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.814 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:04:06.703 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.815 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:04:06.588 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:53.831 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:04:06.605 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:54.005 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:04:06.894 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:54.013 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:04:06.900 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:16:54.122 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:04:06.896 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.855 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:05:09.743 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.877 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:05:09.766 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.887 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:05:09.775 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.890 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:05:09.779 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.911 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:05:09.685 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.920 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:05:09.694 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.941 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:05:09.828 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:56.946 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:05:09.835 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:17:57.006 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:05:09.779 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.755 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:09:16.644 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.761 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:09:16.650 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.770 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:09:16.658 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.776 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:09:16.664 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.785 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:09:16.557 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.789 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:09:16.562 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.927 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:09:16.816 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:03.979 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:09:16.752 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:22:04.017 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:09:16.905 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.501 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:13:31.390 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.517 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:13:31.406 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.543 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:13:31.431 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.544 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:13:31.432 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.561 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:13:31.335 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.599 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:13:31.372 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.651 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:13:31.539 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.716 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:13:31.604 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:26:18.841 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:13:31.615 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.749 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:14:26.629 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.749 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:14:26.635 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.760 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:14:26.647 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.767 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:14:26.654 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.772 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:14:26.546 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.781 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:14:26.554 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.817 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:14:26.705 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.835 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:14:26.716 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:27:13.902 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:14:26.676 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.326 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:16:27.214 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.363 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:16:27.251 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.389 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:16:27.277 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.390 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:16:27.278 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.435 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:16:27.209 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.442 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:16:27.216 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.447 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:16:27.336 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.472 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:16:27.360 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:29:14.541 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:16:27.314 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.035 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:19:37.923 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.043 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:19:37.932 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.055 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:19:37.943 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.068 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:19:37.956 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.075 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:19:37.849 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.092 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:19:37.866 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.181 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:19:38.069 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.201 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:19:38.077 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:32:25.334 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:19:38.107 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.605 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:21:53.492 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.608 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:21:53.497 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.686 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:21:53.574 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.692 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:21:53.466 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.694 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:21:53.582 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.697 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:21:53.471 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.712 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:21:53.601 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.790 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:21:53.677 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:34:40.897 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:21:53.664 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.244 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:24:49.133 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.254 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:24:49.143 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.267 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:24:49.155 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.272 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:24:49.159 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.279 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:24:49.052 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.289 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:24:49.062 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.367 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:24:49.256 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.446 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:24:49.332 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:37:36.540 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:24:49.313 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.394 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:02.282 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.402 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:02.291 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.411 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:02.298 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.414 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:02.302 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.417 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:02.191 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.421 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:02.196 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.586 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:02.475 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.587 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:02.475 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:42:49.643 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:02.417 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.185 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:15.073 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.191 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:15.080 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.227 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:30:15.116 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.235 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:15.123 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.240 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:15.128 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.266 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:15.040 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.270 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:30:15.159 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.275 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:15.049 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:43:02.299 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:30:15.073 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.778 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:35:12.666 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.783 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:35:12.672 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.793 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:35:12.681 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.801 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:35:12.688 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.812 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:35:12.577 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.838 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:35:12.612 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.953 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:35:12.842 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:47:59.968 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:35:12.856 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:48:00.164 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:35:12.929 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.912 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:38:36.801 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.918 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:38:36.808 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.951 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:38:36.839 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.974 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:38:36.862 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.993 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:38:36.767 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:23.996 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:38:36.770 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:24.048 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:38:36.931 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:24.077 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:38:36.965 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:51:24.167 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:38:36.941 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.913 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:40:24.802 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.921 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:40:24.811 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.933 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:40:24.821 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.938 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:40:24.826 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.939 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:40:24.714 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:11.943 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:40:24.717 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:12.011 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:40:24.900 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:12.030 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:40:24.918 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:53:12.064 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:40:24.839 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.746 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:45:37.635 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.753 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:45:37.642 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.754 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:45:37.643 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.756 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:45:37.645 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.784 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:45:37.559 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.813 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:45:37.580 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.940 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:45:37.828 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:24.971 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:45:37.861 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:58:25.116 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:45:37.874 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.688 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:46:58.577 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.702 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:46:58.591 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.749 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:46:58.638 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.756 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:46:58.645 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.761 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:46:58.650 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.764 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:46:58.538 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.779 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:46:58.553 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.803 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:46:58.692 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T20:59:45.848 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:46:58.622 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.708 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:50:47.597 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.716 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:50:47.605 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.727 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:50:47.615 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.730 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:50:47.619 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.744 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:50:47.518 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.758 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:50:47.532 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.929 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:50:47.817 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:34.949 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:50:47.838 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:03:35.010 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:50:47.784 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.352 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:20.241 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.401 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:20.289 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.402 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:20.291 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.414 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:20.304 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.430 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:20.203 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.439 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:20.213 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.524 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:20.412 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.540 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:20.430 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:07.613 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:20.388 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.805 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:57.694 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.812 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:57.700 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.827 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:57.715 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.829 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:57.718 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.832 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:57.607 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.839 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:57.613 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.862 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:55:57.751 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:44.876 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:55:57.765 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:08:45.010 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:55:57.784 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.119 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:57:41.009 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.160 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:57:41.049 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.179 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:57:41.062 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.179 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:57:41.065 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.236 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:57:41.010 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.238 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:57:41.013 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.262 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:09:57:41.152 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.272 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:09:57:41.160 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:10:28.354 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:09:57:41.129 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.050 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:01:07.939 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.078 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:01:07.942 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.078 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:01:07.954 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.078 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:01:07.957 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.082 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:01:07.856 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.092 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:01:07.867 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.196 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:01:08.084 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.223 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:01:08.112 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:13:55.283 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:01:08.057 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.408 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:03:42.297 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.457 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:03:42.345 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.466 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:03:42.355 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.468 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:03:42.357 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.490 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:03:42.265 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.504 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:03:42.278 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.535 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:03:42.424 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.543 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:03:42.431 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:16:29.611 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:03:42.386 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.063 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:06:19.952 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.066 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:06:19.956 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.074 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:06:19.849 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.079 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:06:19.854 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.080 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:06:19.969 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.083 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:06:19.972 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.176 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:06:20.062 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.188 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:06:20.077 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:19:07.193 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:06:19.968 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:20.979 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:11:33.863 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:20.989 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:11:33.872 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:20.991 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:11:33.878 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:20.994 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:11:33.883 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:21.004 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:11:33.779 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:21.020 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:11:33.795 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:21.164 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:11:34.053 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:21.200 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:11:34.089 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:21.308 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:11:34.083 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.071 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:12:08.958 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.074 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:12:08.963 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.117 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:12:09.005 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.118 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:12:09.007 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.147 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:12:08.922 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.149 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:12:08.924 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.158 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:12:09.047 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.162 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:12:09.050 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:24:56.215 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:12:08.990 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.060 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:16:44.945 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.065 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:16:44.955 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.074 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:16:44.963 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.077 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:16:44.966 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.095 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:16:44.871 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.125 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:16:44.900 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.246 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:16:45.135 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.265 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:16:45.155 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:29:32.385 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:16:45.160 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.743 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:20:35.632 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.748 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:20:35.638 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.799 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:20:35.687 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.802 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:20:35.691 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.819 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:20:35.594 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.827 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:20:35.602 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.869 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:20:35.759 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:22.913 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:20:35.801 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:33:23.048 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:20:35.823 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.082 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:21:55.972 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.100 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:21:55.989 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.101 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:21:55.990 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.113 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:21:55.996 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.129 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:21:55.902 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.139 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:21:55.914 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.182 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:21:56.072 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.185 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:21:56.074 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:34:43.288 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:21:56.063 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.758 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:27:07.648 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.770 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:27:07.660 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.777 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:27:07.666 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.779 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:27:07.669 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.788 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:27:07.563 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.798 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:27:07.573 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.963 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:27:07.852 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:54.968 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:27:07.858 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:39:55.122 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:27:07.897 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.371 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:28:57.261 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.379 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:28:57.269 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.415 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:28:57.191 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.422 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:28:57.197 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.426 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:28:57.315 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.437 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:28:57.326 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.450 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:28:57.340 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.499 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:28:57.388 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:41:44.524 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:28:57.299 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.148 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:32:18.038 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.156 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:32:18.046 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.181 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:32:18.064 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.181 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:32:18.070 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.181 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:32:17.956 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.194 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:32:17.970 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.322 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:32:18.212 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.340 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:32:18.231 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:45:05.371 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:32:18.146 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.050 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:18.940 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.060 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:18.950 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.125 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:19.010 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.128 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:19.018 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.144 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:18.920 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.148 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:18.923 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.263 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:19.153 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.291 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:19.181 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:06.365 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:19.140 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.164 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:29.054 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.177 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:29.067 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.187 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:29.077 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.192 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:28.967 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.196 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:29.084 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.203 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:28.978 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.210 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:37:29.100 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.225 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:37:29.115 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:50:16.261 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:37:29.036 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.721 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:38:39.611 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.754 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:38:39.644 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.778 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:38:39.554 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.794 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:38:39.684 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.796 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:38:39.686 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.811 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:38:39.587 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.831 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:38:39.721 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.862 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:38:39.751 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:51:26.891 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:38:39.666 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.631 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:42:41.521 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.641 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:42:41.531 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.660 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:42:41.549 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.667 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:42:41.556 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.667 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:42:41.442 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.696 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:42:41.471 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.826 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:42:41.717 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.854 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:42:41.744 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:55:28.879 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:42:41.654 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.031 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:45:40.922 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.043 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:45:40.933 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.087 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:45:40.976 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.088 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:45:40.978 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.120 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:45:40.895 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.125 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:45:40.901 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.168 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:45:41.058 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.176 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:45:41.066 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T21:58:28.273 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:45:41.049 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.582 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:47:51.472 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.599 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:47:51.489 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.618 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:47:51.508 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.619 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:47:51.394 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.623 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:47:51.512 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.640 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:47:51.406 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.747 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:47:51.638 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.748 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:47:51.638 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:00:38.843 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:47:51.618 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.126 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:53:04.016 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.155 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:53:04.023 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.155 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:53:04.042 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.156 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:53:03.931 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.157 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:53:04.046 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.162 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:53:03.937 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.314 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:53:04.205 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.363 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:53:04.253 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:05:51.458 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:53:04.231 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.690 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:54:02.581 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.700 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:54:02.591 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.730 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:54:02.620 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.732 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:54:02.621 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.738 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:54:02.514 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.741 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:54:02.517 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.789 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:54:02.680 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.791 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:54:02.681 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:06:49.827 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:54:02.603 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.725 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:58:14.615 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.740 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:58:14.624 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.749 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:58:14.638 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.754 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:58:14.644 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.756 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:58:14.531 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.798 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:58:14.573 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:01.886 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:10:58:14.776 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:02.029 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:10:58:14.919 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:11:02.068 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:10:58:14.843 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.443 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:02:24.333 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.468 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:02:24.358 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.490 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:02:24.380 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.537 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:02:24.426 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.542 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:02:24.318 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.547 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:02:24.323 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.599 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:02:24.490 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.670 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:02:24.559 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:15:11.712 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:02:24.487 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.564 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:03:26.452 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.568 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:03:26.459 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.578 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:03:26.468 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.582 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:03:26.472 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.595 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:03:26.371 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.622 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:03:26.398 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.630 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:03:26.521 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.639 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:03:26.529 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:16:13.741 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:03:26.516 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.774 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:08:38.664 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.783 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:08:38.674 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.795 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:08:38.684 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.800 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:08:38.690 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.814 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:08:38.590 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.826 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:08:38.602 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:25.990 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:08:38.880 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:26.037 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:08:38.927 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:21:26.121 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:08:38.897 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.040 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:10:45.931 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.073 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:10:45.964 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.113 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:10:45.889 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.116 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:10:45.892 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.168 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:10:46.059 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.203 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:10:46.093 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.230 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:10:46.120 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.251 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:10:46.027 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:23:33.293 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:10:46.183 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:37.968 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:13:50.859 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:37.977 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:13:50.868 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:37.985 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:13:50.875 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:37.990 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:13:50.880 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:37.995 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:13:50.770 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:38.006 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:13:50.782 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:38.175 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:13:51.064 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:38.177 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:13:51.067 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:26:38.191 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:13:50.967 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.728 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:02.618 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.731 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:02.622 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.758 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:02.534 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.785 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:02.547 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.865 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:02.755 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.872 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:02.762 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:49.915 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:02.806 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:50.003 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:02.779 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:50.166 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:03.056 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.735 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:07.626 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.749 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:07.640 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.780 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:07.671 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.790 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:07.680 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.802 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:07.691 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.824 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:07.714 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.829 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:07.604 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.832 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:07.608 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:31:54.870 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:07.646 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.199 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:38.090 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.208 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:38.098 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.209 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:38.099 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.241 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:38.131 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.260 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:38.036 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.262 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:19:38.153 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.271 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:38.047 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.275 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:19:38.165 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:32:25.333 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:19:38.108 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.372 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:24:13.263 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.399 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:24:13.273 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.399 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:24:13.280 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.399 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:24:13.284 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.399 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:24:13.175 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.409 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:24:13.185 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.550 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:24:13.440 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.551 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:24:13.442 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:37:00.667 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:24:13.443 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.830 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:27:29.720 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.852 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:27:29.730 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.852 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:27:29.739 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.897 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:27:29.787 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.921 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:27:29.697 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:16.924 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:27:29.700 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:17.001 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:27:29.889 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:17.060 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:27:29.950 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:40:17.134 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:27:29.910 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.249 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:29:25.140 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.255 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:29:25.146 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.266 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:29:25.156 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.270 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:29:25.161 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.281 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:29:25.057 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.308 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:29:25.084 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.381 INFO:tasks.ceph.mon.a.smithi069.stderr:==00:11:29:25.272 8772== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.429 INFO:tasks.ceph.mon.c.smithi069.stderr:==00:11:29:25.279 8771== Warning: unimplemented fcntl command: 1036 2019-11-29T22:42:12.465 INFO:tasks.ceph.mon.b.smithi163.stderr:==00:11:29:25.241 8398== Warning: unimplemented fcntl command: 1036 2019-11-29T22:46:23.831 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-11-29T22:46:23.831 DEBUG:teuthology.task.console_log:Killing console logger for smithi163 2019-11-29T22:46:23.832 DEBUG:teuthology.task.console_log:Killing console logger for smithi069 2019-11-29T22:46:23.832 DEBUG:teuthology.task.console_log:Killing console logger for smithi163 2019-11-29T22:46:23.832 DEBUG:teuthology.task.console_log:Killing console logger for smithi069 2019-11-29T22:46:23.833 DEBUG:teuthology.exit:Finished running handlers